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Abstract. A generalization of the Macdonald polynomials depending upon both commuting and anti- 
commuting variables has been introduced recently. The construction relies on certain orthogonality and 
triangularity relations. Although many superpolynomials were constructed as solutions of highly over- 
determined system, the existence issue was left open. This is resolved here: we demonstrate that the 
underlying construction has a (unique) solution. The proof uses, as a starting point, the definition of the 
Macdonald superpolynomials in terms of the Macdonald non-symmetric polynomials via a non-standard 
(anti)symmetrization and a suitable dressing by anticommuting monomials. This relationship naturally 
suggests the form of two family of commuting operators that have the defined superpolynomials as their 
■ common eigenfunctions. These eigenfunctions are then shown to be triangular and orthogonal. Up 

to a normalization, these two conditions uniquely characterize these superpolynomials. Moreover, the 
Macdonald superpolynomials are found to be orthogonal with respect to a second (constant-term-type) 
scalar product and its norm is evaluated. The latter is shown to match (up to a g-power) the conjectured 
norm with respect to the original scalar product. Finally, we recall the super-version of the Macdonald 
positivity conjecture and present two new conjectures which both provide a remarkable relationship 
between the new (g,t)-Kostka coefficients and the usual ones. 



1. Introduction 

1.1. Macdonald superpolynomials and related positivity conjectures. A candidate for the su- 
perspace extension of the Macdonald polynomials - Macdonald superpolynomials for short - has been 
obtained in [7J. Such an extension involves the anticommuting variables 0i,92,--- (with 0f = 0) as well 
as the usual commuting variables xi,X2, ■ ■ ■ The superspace approach turns out to be a very restrictive 
framework: each variable Xi is considered to be paired with an anticommuting variable 0^, so that sym- 
metric superpolynomials are required to be invariant under the interchange of pairs {xi,9i) O {xj,9j) 

The construction in [J is presented as a conjecture (a point developed shortly). But the first exploration 
of the resulting superpolynomials revealed a very rich structure. As expected, this two-parameter (q, t) 
family of superpolynomials contains, in the appropriate limit (q = t", t — )> 1), the Jack superpolynomials 
[lOj . But, what was totally unexpected a priori, is that it contains two versions of the Hall-Littlewood 
superpolynomials (for g = and q — oo). Moreover, from each version of the latter, we can define a 
natural extension of the Schur polynomials (for t = and i — > oo respectively) . Both types of Schurs have 
a positive integral decomposition into the monomial superpolynomials (a property that is not verified for 
the Jack superpolynomials at a = 1). But what is more remarkable is that, using the q — t = Schurs, 
we could conjecture a generalization of the Macdonald positivity conjecture (reviewed below). 

The present article, although completely independent, is a continuation of our previous work [7]. It 
addresses the following issues: the existence of the Macdonald superpolynomials, their relation with the 
non-symmetric Macdonald polynomials and their characterization as an eigenvalue problem. As an aside, 
new conjectures for the Kostka coefficients are presented. 
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1.2. The main result: an existence proof. The conjectural construction of [7^ is proved here to be a 
valid characterization of the Macdonald polynomials in superspace. To clarify this point, we first recall 
the definition of the ordinary Macdonald polynomials ^2^. 

The Macdonald polynomials P\ = P\{x; q, t), in the variables x = Xi,X2, ■ ■ ■ , are characterized by the 
two conditions 

1) Px = nix + lower terms, 

(1.1) 

2) {{Px\P^))g,t^O if A^/i- 

The triangular decomposition refers to the usual dominance order on partitions (see (|2.6p below) and 
the toa's are the monomial symmetric functions. The orthogonality relation is defined in the power-sum 
basis Px = PAi ■ ■ -PXi, with pr = J2i>i < , 



as 



1 — q 



{{Px\p^)),,t = z^S^,l[j-^ where za = J] ' (l'^) 



i=l i>l 



nx{i) being the number of parts in A equal to i. Since the dominance ordering is partial (for degrees 
> 6), the orthogonality constraint leads to an overdetermined system at each degree > 6. It is therefore 
necessary to show that there exists such a family of polynomials. This is generally done through an 
eigenvalue-problem characterization. 

The brute-force approach followed in [J was to look for a suitable deformation of the scalar product 
(II. 2p that allows for nontrivial solutions to these systems. In this way, a candidate scalar product was 
identified and a large number of Macdonald superpolynomials were constructed. The correctness of 
the construction was corroborated by various conjectural properties that provide natural extensions to 
superspace of classical results on Macdonald polynomials. However, establishing the existence of the 
Macdonald superpolynomials remained an open problem, the corresponding eigenvalue problem being 
still missing. 

The existence issue is resolved here: it is demonstrated that the superspace extension of the criteria 
(II. ip has a solution. This is our main result (the notation that follows is explained in full detail in Section 
[2]). 

Theorem 1. Given a superpartition A = (A"; A'*) of fermionic degree m, there is a unique symmetric 
superpolynomial Pa = Ph{x, 0; q, t), with x — (xi, a;2, . . . ) and 9 — {9i,02, ■ ■ ■), such that: 

1) Pa = "^A + lower terms, 

(1.3) 

2) {{PA\Pn))q,t^O if A^n, 

where toa is a monomial superpolynomial and where lower terms refer to the dominance ordering on 
superpartitions (see i2.5]) ). The scalar product is defined by 

{{PA\pn))q,t = (-1)(") ZA{q,t)SAn, where ZA{q,t) = za= g'^"' H i J^M ■ (l-^) 

i—l 

1.3. A key relationship: the connection with Macdonald non-symmetric polynomials. The 

existence proof proceeds indirectly, via an alternative definition of the Macdonald superpolynomials, this 
one relying on a suitable (anti)symmetrization of the non-symmetric Macdonald polynomials. Such a 
construction is akin to that of the Jack superpolynomials in terms of the non-symmetric polynomials 
worked out in jlOl Sect. 9]. However, the present construction turns out to be trickier than in the Jack 
case. Indeed, Macdonal polynomials with prescribed symmetry - a priori, expected building blocks - 
are obtained by i-(anti)symmetrization of some subset of variables, where the role of symmetric group is 
played by the Hecke algebra [ll|3]. However, anticommuting variable cannot be i-antisymmetrized, and 
the construction has to incorporate both the usual antisymmetrization and the i-symmetrization, the two 
operations being applied to distinct set of variables. This is made explicit in eq. (|4.ip . 
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This definition of tlie Macdonaid superpolynomiais in terms of non-symmetric Macdonald polynomials 
is a crucial step toward the characterization of the former in terms of an eigenvalue problem. Recall 
that these non-symmetric polynomials are eigenfunctions of the Cherednik operators [5]. The mere 
symnietrization procedure that defines the Macdonald superpolynomiais indicates how to dress symmetric 
combinations of the Cherednik operators in order to generate operators whose eigenfunctions are the 
Macdonal superpolynomiais. Because we have two sets of variables (commuting and anticommuting) , we 
need two families of commuting operators to obtain a non-degenerate eigenvalue characterization (see 
e.g. [TU] for the Jack case). We thereby construct two generating functions of commuting operators. 
These are the natural extension of the Sekiguchi operators characterizing the Jack superpolynomiais and 
introduced in [Ml Sect. 3]. To fully characterize the eigenvalue problem, it is sufficient to consider the 
simplest representative of each family. 

The proof of the existence of the superpolynomiais defined by the two conditions in Theorem[T]proceeds 
along standard arguments. But a crucial and difficult result that needs to be established is the self-adjoint 
property of the two eigenoperators. The outline of this proof was provided to us by Alain Lascoux |21j . 

The relationship between Macdonald superpolynomiais and the non-symmetric polynomials has a 
further direct consequence: it implies a second orthogonality relation, where in this case, the scalar 
product is a constant-term expression. We show that when the number of variables tends to infinity, the 
norm calculated from the constant-term scalar product is equal (up to a g-power) to the norm, conjectured 
in [7], with respect to the scalar product defined in Theorem [TJ 

1.4. Organization of the article. The outline of the article is the following. In Section [21 we recall 
some basic definitions related to superpolynomiais and superpartitions. Section [3] is also devoted to 
a review of known results, here pertaining to non-symmetric Macdonald polynomials. The definition 
of the Macdonald superpolynomiais in terms of non-symmetric Macdonald polynomials is presented in 
Section 14.11 In Section 14.21 we introduce two generating functions for commuting operators and show 
that they have the Macdonald superpolynomiais as their common eigenfunctions. This eigenfunction 
characterization allows us to demonstrate in Section 14.31 that the Macdonald superpolynomiais have a 
triangular decomposition in the monomial basis. In preparation for the orthogonality proof, the eigenvalue 
problem defining the Macdonald superpolynomiais is simplified in Section 14.41 where it is shown to be 
sufficient to consider two eigenoperators to get a non-degenerate characterization. The proof of the 
orthogonality with respect to the scalar product (11.41) . which solves the existence issue in Theorem [1] is 
worked out in SectionjS] At first, the problem is reformulated in terms of action of the two eigenoperators 
on the kernel. The long proof of the self-adjoint property of the Macdonald-type operators is worked 
out in Section 15.21 As a natural extension of these orthogonality results, a non-trivial duality relation 
is established in Section Given this, all the tools for the demonstration of the evaluation and norm 
conjectures of [7] are available, proof that would follow that in [13] for the Jack superpolynomiais. Finally, 
a second orthogonality relation is demonstrated in Section |6l The corresponding norm is evaluated, and 
shown to be equal, up to a power of q, to the conjectural norm of the Macdonald superpolynomiais 
with respect to the original scalar product. The combinatorial identity on which this claim relies is 
demonstrated in the Appendix. 

The last section (Section[7]), devoted to the generalized Kostka coefficients, is somewhat off the stream- 
line of this article but should be viewed in the context of the continuation of [7] . The superspace version 
of the Macdonald positivity conjecture is recalled in Section Ffm Two symmetry properties of the general- 
ized Kostka coefficients are presented. Although these are rather direct extensions of the usual symmetry 
relations given in [23j . one of these involve a new combinatorial number that is specific to superpartitions. 
This new data illustrates well the kind of novelties brought in by the introduction of anticommuting vari- 
ables and the richness of the combinatorics of superpartitions. Tables of Kostka coefficients are appended 
to this section. 

Two new conjectures concerning the Kostka coefficients are given in Section [7.21 Both results exhibit 
a different relationship between the (q, t)-Kostka coefficients in the m = 1 sector with the ordinary (i.e.. 
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TO = 0) {q, t)-Kostka coefficients. In particular, Coniecture[25lexpresses the usual {q, t)-Kostka coefficients 
for partitions of degree n as a sum of (q, t)-Kostka coefficients for superpartitions of degree n — 1 and 
TO = 1. In other words, the super- version of the Macdonald positivity conjecture provides a refinement 
of the usual ones. Let us digress briefly and point out that this result naturally poses the question: to 
which extent could we get information on the usual (q, t)-Kostka coefficients from the perspective of the 
Macdonald superpolynomials? We intend to investigate this point elsewhere from a particular angle. Such 
a connexion, at this stage, could rightly be called "Science fiction and Macdonald's superpolynomials" 

2. Symmetric polynomials in superspace 

A polynomial in superspace, or equivalently, a superpolynomial, is a polynomial in the usual N vari- 
ables xi,-- - ,xn and the N anticommuting variables 6i,--- ,9n over a certain field, which will be 
taken throughout this article to be Q{q,t). A superpolynomial P{x,0), with x = (xi, . . . ,xn) and 
6 = ((?!,..., 9n), is said to be symmetric if [5]: 

JC^Pix, 9) = P{x, 9) for all a & Sn , (2.1) 

where 

K.^n.K^, with I : (2.2) 

y Kcr : {9i, . . . ,9n) ^ {9a(l)., ■ ■ ■ ■,9cr(N))- 

The space of symmetric superpolynomials in N variables over the field Q{q,t) will be denoted ^at, and 
its inverse limit by ^ (loosely speaking, the number of variables is considered infinite in 

Before defining superpartitions, we recall some definitions related to partitions [23]. A partition A = 
(Ai, A2, . . . ) of degree |A| is a vector of non- negative integers such that A^ > Aj+i for i = 1,2,... and 
such that = I'^l- The length f (A) of A is the number of non-zero entries of A. Each partition A has 

an associated Ferrers diagram with A^ lattice squares in the i*'* row, from the top to bottom. Any lattice 
square in the Ferrers diagram is called a cell (or simply a square), where the cell (i,j) is in the ith row 
and jth column of the diagram. The conjugate A' of a partition A is represented by the diagram obtained 
by reflecting A about the main diagonal. Given a cell s = in A, we let 

aA(s) = Aj-j, and lx{s) ^ \'j ~ i . (2.3) 

The quantities a\{s) and l\{s) are respectively called the arm-length and leg- length. We say that the 
diagram /j, is contained in A, denoted C A, if /x^ < A^ for all i. Finally, A//x is a horizontal (resp. 
vertical) n-strip if /i C A, |A| — = n, and the skew diagram A//z does not have two cells in the same 
column (resp. row). 

Symmetric superpolynomials are naturally indexed by superpartitions [S] . A superpartition A of degree 
(n\m) and length ^ is a pair (A®, A*) of partitions A® and A* such that [T^ : 

(1) A* C A® 

(2) the degree of A* is n 

(3) the length of A® is I 

(4) the skew diagram A®/A* is both a horizontal and a vertical TO-strip. 

We refer to to and n respectively as the fermionic degree and total degree of A. Obviously, if A® = A* = A, 
then A — (A, A) can be interpreted as the partition A. 

We will also need another characterization of a superpartition. A superpartition A is a pair of partitions 
(A"; A*) = (Ai, . . . , Am,; Am+i, . . . , Ajv), where A" is a partition with to distinct parts (one of them 
possibly equal to zero), and A* is an ordinary partition. The correspondence between (A®, A*) and 
(A°; A*) is given explicitely as follows: given (A®, A*), the parts of A" correspond to the parts of A* such 
that A® ^ A* , while the parts of A*" correspond to the parts of A* such that A® = A* . 
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The conjugate of a superpartition A = (A®, A*) is A' = ((A®)', (A*)'). A diagrammatic represen- 
tation of A is given by the Ferrers diagram of A* with circles added in the cells corresponding to 
A® /A*. For instance, we have, for A = (A^-A") = (3, 1,0; 2,1) (or equivalently A = (A®, A*) = 
((4,2,2,1,1),(3,2,1,1))): 



A 



D 



A® : 



A* 



A' 



D 



(2.4) 



where the last diagram illustrates the conjugation operation that corresponds, as usual, to replacing rows 
by columns. 



The extension of the dominance ordering to superpartitions is |13j : 

n<A iff deg(A) = deg(rj), ft* < A* and n® < A^' 



(2.5) 



Note that comparing two superpartitions amounts to comparing two pairs of ordinary partitions, (il* 
A*) and (£7®, A®), with respect to the usual dominance ordering: 



/i < A iff \^J'\ = |A| and Mi + ••• + /ii < Ai + ■•• + Vi . 



(2.6) 



Two simple bases of the space of symmetric polynomials in superspace will be particulary relevant to 
our work. The extension of the monomial symmetric functions, ttia — m\{x,9), defined by: 



rriA 



- ie^---e,nx^' ■■■X 



(2.7) 



where the sum is over distinct permutations, and the generalization of the power-sum symmetric functions 
PA ^Pa{x,0): 



PA = PAi • • -PA^PA^+i • • -PAjv, 

with fc > and r > 1. 



where pk = OiX^ and Pr = : 



(2.8) 



3. The non-symmetric Macdonald polynomials 

The ordinary Macdonald polynomials can be defined by the conditions (1) and (2) in But they 

could alternatively be defined directly in terms of the so-called non-symmetric Macdonald polynomials 
by a suitable symmetrization process [231 H] (see also [151 [55]). As will be shown in the following section, 
this can also be done for their superpace extension. But since this result uses a fair amount of notation 
and definitions, it is convenient to summarize these here. 

The non-symmetric Macdonald polynomials are defined in terms of an eigenvalue problem formulated 
in terms of the Cherednik operators [8J. They are constructed from the operators T,; defined as 

T^,=i+^^^3^(ifM+i-l), *-l,...,iV-l, (3.1) 

Xi Xi-^-l 

where we recall that Ki^i^i exchanges the variables Xi and x^+i. Note that for t = 1, Ti reduces to Ki^i^i. 
The Ti's satisfy the Hecke algebra relations: 



{T, - t){T, + 1) = 

J-iJ-i+l-l-i — J-i+l-l-iJ-i+l 

T.Tj^TjT, \i^]\>l. 



(3.2) 
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To define the Cherednik operators, we also need to introduce the g-shift operators 

Xj tH- Xj if j ^ i, 



(3.3) 



and the operator uj defined as: 

io ^ Kn-i,n ■ ■ ■ Ki^2Ti. (3.4) 
We note that wT, = T,_ia; for i = 2, . . . , TV - 1. 

We are now in position to define the Cherednick operators: 

Y, = t-^+'T, ■ ■ • Tn-iujT^-' ■ ■ ■ T-\, (3.5) 

where Tj'^ (also denoted Tj below) is 

T-i =f-i - 1 + (3.6) 

which follows from the quadratic relation (|3.2p of the Hecke algebra. These operators satisfy the following 
relations [TT] : 

T,y, = y,+ir, + (t-i)y, 

T,Y,+i ^ Y,T, - (t - l)Y, 

T.Yj^YjT, if J ^1,1 + 1. (3.7) 

It can be easily deduced from these relations that 

(y, + K,+i)T, = i^,(r, + r,+i) and (y,y,+i)T, = T,(y,y,+i). (3.8) 

But more importantly, the y^'s commute among each others, [Yi, Yj] — 0, and can therefore be simultane- 
ously diagonalized. Their eigenfunctions are the (monic) non-symmetric Macdonald polynomials (labeled 
by compositions). To be more precise, the non-symmetric Macdonald polynomial E^i is the unique poly- 
nomial with rational coefficients in q and t that is triangularly related to the monomials (in the Bruhat 
ordering on compositions) 

E^ = x'^ + J2 ^'f'^" (3.9) 

and that satisfies, for all i — 1, . . . ,N, 

Y,E^=fj,E,j, where fj, = q'^H-'-^'^ (3.10) 

with — #{fc < i\rjk > rji} + ^{k > i\rik > Vi}- The Bruhat order on compositions is defined as 
follows: 

V ^ rj if < 7]^ or = 7]^ and w^j < Wjy, (3.11) 

where ry"*" is the partition associated to rj and Wj^ is the unique permutation of minimal length such that 
r] = Wri'T]'^ (wri pcrmutcs the entries of ri~^). In the Bruhat order on the symmetric group, < Wi, iff Wr/ 
can be obtained as a subword of w^. 

The following two properties of the non-symmetric Macdonald polynomials will be needed below. The 
first one expresses the stability of the polynomials i?^ with respect to the number of variables (see e.g. 
[H eq. (3.2)]): 

n^ ( Ejj__{xi,...,xn-i) if r]N = ro io\ 

£;,,(xi,...,a;7v_i,0) = j ^' ^ if ^ ^ ' 
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where rj- — (771, ... , r/N-i)- The second one gives the action of the operators Ti on Erf (see e.g. [3l eq. 
(8)] and 2,): 

^YT^^ + *^s,v if < Vi+i 

T,E^ = <! tS„ if = 77,+i (3.13) 

where Si^^ = fjjf]i+i and s^i] = (r/i, . . . , r/i_i, 77^+1, ry^, ?7j+2, ■ • ■ , w)- 

Finally, we introduce the i-symnietrization and t-antisynimetrization operators of variables xi, . . . , xn 

Un^ ^""^ ^N= J2 {-^y'^^^T^ (3-14) 

where 

Ta^Ti^...Ti^ if cr = . . . . (3.15) 
Note that for any polynomial / in the variables xi, . . . , xm, we have Ki^i+iU^f — U^f, but Ki^i^iUj^f / 
-[/j^/ since [261 eq.(2.26)]: 

= where = V (-1)'^"^^^. (3.16) 

(i.e., is the usual antisymmetrization operator). stands for 

n (^2;, -.xj) and AateeA^v- (3.17) 

l<i<j<N 

Below, we will designate by Sm and Sm" the group of permutations of the variables xi, . . . , Xm and 
Xm+i, ■ ■ ■ , xn respectively. For instance, ?7~ and U^c are defined as in p.l4p but with Sn replaced by 
Sm and Sm" respectively. Similarly, we will frequently use the notation A^ which is defined as in (j3.17p 
but with replaced by m. 

4. Macdonald superpolynomials 

4.1. Definition of the Macdonald superpolynomials. We are now in position to define the Mac- 
donald superpolynomials in terms of the non-symmetric Macdonald polynomials. We will prove later 
that the Macdonald superpolynomials defined in the next definition do in fact provide a solution to the 
existence problem in Theorem [TJ 

Definition 2. The Macdonald superpolynomials P\ are defined as 

PA^PA{x,e;q,t)^ m\^iMA^) E ICa9l---9mAmU+.EAi^, (4.1) 

where 

/A.(i) = n["A=W]*!' (4.2) 

with riA" (j) being the number of occurrences of j in A* and stands for the concatenation of A" and 
A" read in reverse order: A^ — (Am, . . . , Ai, Aat, . . . , Am+i)- 



In ()4.ip and (|4.2p we used the standard notation: 

[k]t\ = [lU2]f-[k]t with [m]t = (l-t™)/(l-<). 

We also extended the usual concept of inversion on a permutation to a partition: inv(A'') is the number 
of inversions in A^ , where this latter number is 

inv(A) =#{n>z> j|A, < Aj}, (4.3) 
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where n is the number of entries in A (including O's). For instance, we have inv(22100) = 8. 

We first show that the stabihty of E^j with respect to the number of variables can be lifted to that of 
Pa- 

Proposition 3. Suppose that N > m. Then the Macdonald superpolynomials P\ are stable with respect 
the number of variables, that is, 



Pa_ {xi,...,xn- 




7N- 



PaIxi,. . . ,a;jv-i, 0,6*1, . . . ,6'Ar_i,0) = 
where A_ = (Ai, . . . , A™; A^+i, . . . , Aat-i). 

Proof. From the definition of Pa it is immediate that it suffices to show that 

' ^C/+._i?A« ifAAT^O 

^0 if Aat 7^ ' 

where ml refers to the reduced set of variables Xm+i, ■ ■ ■ , xn-i and 

A^ = (A^)-" = (A„, . . . , Ai, Ajv-i, . . . , A„+i). 



ifAw = 
if Ajv 7^ 



(4.4) 



(4.5) 



(4.6) 



We stress that although A_ is a superpartition, A;'^ is a composition. The constant CA(t) is the normal- 
ization constant in the definition (14.11): 



CA{t) 



fAs{t) ti"v(A= 



Now U+a can be factorized as follows: 



U.^^a — U^a (1 + Tn-i + Tn-iTn-2 



Tn-1 ■ ■ ■ Tm+l), 



so that 



xjv-u - L J xn = 

Now, if A* has exactly k = n^^ (0) zero entries, it is easy to see from p.l2p and p.l3p that 

Tn-1 • • • Tm+iE^R 



XN=0 1 



^N-m-t^^^ if i < fc 



if i > fc. 



Hence 



[U+.Ear]^^^„ 



N — m-k!!,] TJ+ 



'[k]tU+.Ej^R. iffc>0 
if fc = 0. 



(4.7) 

(4.8) 
(4.9) 

(4.10) 
(4.11) 



If fc > we have that inv(Ai) ~ inv(A*) — A^+rn+fc. Therefore t ™ [k]t — ca_/ca and the proposition 



follows. 



□ 



4.2. Two family of commuting (eigen)operators. We now introduce two families of operators gen- 
eralizing those introduced for the Jack superpolynomials [HI Sect. 3] and defined as 



N 



N 



and 



m=Oo-£S„/(S,„xS„.c) \ ™ 1=1 



D®{u-q,t)=Yl E ^A-:^\{{l + uqY,) [] (l + ur.)7r^^i. 



(4.12) 



(4.13) 
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The operator 7ri,...,m is the projection operator defined as 

m N 

X{e,de^ [] dg^Oj. (4.14) 

2—1 J— m+1 

It is easy to see that 

n iOl---Ora if / = {1, . . . , m} 

1^0 if / 7^ {1, . . 

where 9i ^ .. . 9^,^ if / = {ii, . . . ,ik}. 

If we let £>* (resp. D®) be the coefficient of u" in D*{u;q,t) (resp. D®{u;q,t)), the operators D* 
and I?® can be seen as the generating series of the operators D* and D® respectively. These operators, 
when restricted to act on can be considered as the generalization to superspace of the Macdonald 
operators. 

As will be shown below, the superpolynomials Pa are eigenfunctions of both D*{u; q, t) and D®{u; q, t). 
A rationale for the rather intricate structure of the operators is the following: in order for Pa, as given 
by (|4.ip . to be an eigenfunction of an operator built out of J|(l + uYi), the factor A*„/A„i needs to 
be inserted to the right to transform Am into C/~ via (|3.16p . so that U~ can be commuted through the 
factors J|(l + uYi). Finally, the term A^/A^ is added to the left to retransform U~ into Am- We now 
state the eigenfunction characterization and plunge into the details of the proof. 

Proposition 4. We have 

D*{u;q,t)P^^SA,iu;q,t)Pf, and D® {u; q,t) P^ ^ e^s{u; q,t) Pa , (4.16) 

where e\{u;q,t) is given, for any partition A, by 

N 

ex{u;q,t)^l[{l + uq^'t^~'). (4.17) 

i=l 

Proof. First observe that 

T^l....,lK.w9l^-^9m= \ n +T, ■ (4-18) 

otherwise. 



We thus have 



A A* 

D*{u-q,t)PA= J2 JCa^Y[il+uY)^9i...9mAmU+.EAn 

<tGSn/(S„xS,„c) ™ i=l ™ 

A ^ 

lcJ^)^l[il + uYi)9,...9mUmU+.EAH. (4.19) 

creSN/(S„xS„c) ™ i=l 

Relations p.8p imply that the product IliLill + "^^i) '^^^ tie moved beyond the factors U: 

A ^ 
D*{u;q,t)PA= J2 ICat^'^^^ei...9mUmU+.Y[{l + uY,)EAH 

(TeSiv/(S,„xS„c) ™ i=l 

N 

J2 IC„ei...9mAmU+.Y[{l + uY,)EAH (4.20) 

aeSN/{S,r,>iS„<,} 1=1 

To prove the first statement, it thus suffices to prove that 

^]^(l + uy,)^ EAR=SA,iu;q,t)EAn. (4.21) 
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Similarly, to prove the second statement, we have to prove that 

Cm N \ 

Ylil + uqY,) Y[ {l + uY,)\ E^n =e^e(u;q,t)E^n. (4.22) 
2—1 2— m+1 / 

Let T] — and suppose that rji = r. It is easy to see that the quantity /^(i) in the eigenvalue 
fji = g''t-'^(*) of Y, (see ([XTU]) ) is such that 

— #{rows of A* of size larger than r} + #{rows of of size r above row i\ . (4.23) 

Therefore, letting 

ji = :/^{rows of A* of size larger than r} + ^^{rows of A^ of size r above row i} + 1 (4.24) 
we have {ji, . . . , ja?} = {!,..., TV}, A*. — r and we recover (|4.2ip in the form: 

(TV \ TV 

[](1 + uY,) = n (1 + m^^'i^''')EAH. (4.25) 

i=l / ]i=l 

Continuing with the same notation for the second case (|4.22p . we have that if i belongs to {1, . . . , m} 
then ryi = r is the highest row of size r in 77, and thus by (|4.24p A*^ is also the highest row of size r in 
A*. Hence, the eigenvalue in this case is 

qY.Ej^R = q'^H+'^t^-^^Ej~,R. (4.26) 
But since i e {1, . . . , m}. A*, e A° and therefore A*. + 1 = A®. Now, if i does not belong to {!,..., m}, 
then we have A*. = A® and then: 

Cm N \ N 

n(i + uqYi) n (1 + "^«) = n (1 + v^'^'^'o^a^- (4.27) 
i—l i—m-\-l / ji — ^ 

□ 

4.3. The triangular decomposition of the Macdonald superpolynomials. At this point, we have 
established that the Pa's are eigenfunctions of the operators D*{u;q,t) and D® {u; q,t). We now show 
that this characterization of the Macdonald superpolynomials entails a triangular decomposition into 
monomials. The chosen normalization in (j4.ip will make this decomposition unitriangular. 

It is well known that if x'^ appears in the monomial expansion of YiX^ , then "f < r] [241 Eq. (4.13)]. This 
statement however is not sufficient for our purposes. We now give a slightly more precise characterization 
of the triangular action of the Cherednik operators Yi on monomials. For this, we need to define some 
operations on compositions. Suppose i < j. Let 

s,,j(...,?7i,...,?7j,...) = (...,77j, ...,77,,...). (4.28) 

Let also Sij be the following restriction of Sij: 

Sij{...,T]„...,rij,...) = {...,r]j,...,r]i,...) only if T],>T]j. (4.29) 

Finally, for £ = 1, . . . , [77^ — ryj| — 1, let 

.f](.. .,7y„...) = ( iiv.>V, (4.30) 

'-^ [{... ,m + £,... ,rij -£,...) it T], < rij 

^ (£) . ^ (£) 

Note that we will often use s^, Si, and s] instead of Si^i^-i, s^.i+i and Sj- ,/_(_]^, respectively. 
Lemma 5. Suppose that x'^ occurs in the monomial expansion oJYiX^ . Then 

i = 9i---gAv) (4.31) 

where gk either stands for Si^j^, or s|^^| . We stress that if gt = Si^j^ then uii^ > ujj^ , where lo — 
gk+i ■ --griv)- 
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Proof. One easily shows that 

T,x'> ^a^x^ +b^x''^ + {l-t)sgn{rj,-^,+i) ^ x''''\ (4.32) 

where sgn(a;) is the sign of x and 

a, = ^-^' '^^<'^^+^ and 6, = /'' ^ '^^^^ (4.33) 
[0, r]i > r/i+i 1^1, r]i > rji+i 

Thus, the monomials that appear in TiX^ are of the form x^^^'^'\ x^^^ for I — 1, . . . , jry^ — 77^+1 1 — 1 and, 
possibly an extra x^ . The latter extra term a;'' appears only if -qi < r/i+i. We stress that if 77^ = ?7i+i, 
the operator Si also acts as the identity on -q. However, even in the latter case, we prefer to write SiVj 
explicitly in order to avoid any confusion with the identity operator that produces the very first term on 
the rhs of (|432l) . 

The action of the inverse operator Ti — t^^Tj + {t~^ — 1) immediately follows from ()4.32p . Once more, 
the monomials that appear in Tix"^ are of the form x^'^''-', a;*^ for i = 1, . . . , \rii — r/i+ij — 1 and, 
possibly an extra x"^ . However, the latter extra term x'^ appears this time only if rji > r/i+i. 

Hence, from the definition of Yi, the terms x'^ appearing in YiX^ are such that 

1 = fi--- fN~iSN~i ■ ■ ■ si/i • • • fi-i{v) (4.34) 

where fj and fj correspond either to sj, s^p or the identity (whenever allowed). 

Let J be such that fj does not act as Sj, and suppose that j is the rightmost amongst such terms. We 
have that (note that j < i) 

1 = fi--- fN^iSN^i - - - si/i • • • fj-iSj ■ ■ ■ Si_i(si_i • • • Sj+iSjfjSj+i ■ ■ ■ Si-i{ri)) (4.35) 

Observe that fi ■ ■ - fN-i^N-i - ■ - sifi - - ■ fj-iSj - ■ - Si_i corresponds to the operator that appears in (j4.34l) 
with fk — Sk for k = j, . . . ,i — I. li fj is the identity, we have 

Si-i - ■ - Sj+iSjfjSj+i ■ ■ ■ Si_i(r/) = Sj_i • • • Sj+iSjSj+i ■ ■ ■ Si-i(?7) = Sj^ir]) (4.36) 

Note that by supposition, r]j > rji since fj acted as the identity. If fj — Sj , we have 

Si-i ■ ■ ■ Sj+iSjfjSj+i ■ • •Si_i(?7) = Si_i • • • Sj+iSjS^Sj+i ■ • •Si_i(?7) = sf^\r]) (4.37) 

for some £' (the £' is such that Sj^iS^ = sjV)- Repeating the same process, we can get rid of all fk such 
that fk does not act as Sfc. More explicitly, we have shown that 

1^ fi- -- fN~iSN~\ - --s\f\ ■ • •/i_i(/x), (4.38) 

where all the fk act as Sk and where is the composition obtained from r\ by the action of some sjj and 
Sji with j < i. 

Now suppose that fj acts as the identity and suppose that j is the rightmost such terms. In this case, 
(supposing that all fk act as Sk) (|4.38p becomes (note that i < j this time) 

1 = fi--- fj-ifjSj+i - - - SN-iSN-i - - - sisi ■ ■ ■ Si_i(^) ^ fi--- fj^ifjSj - ■ - Si{fi) (4.39) 

Hence 

1 ^ fi--- f]-iSjSj ■ ■ ■ Si{siSi+i ■ ■ ■ Sj-ifjSjSj-i ■ ■ ■ Si+iSi{fi)) (4.40) 
If fj is the identity, we have 

{siSi+i ■ ■ ■ Sj-ifjSjSj-i ■ ■ ■ Si+iSi{^i)) = {siSi+1 ■ ■ ■ Sj-iSjSj^i ■ ■ ■ Si+iSi{fj,)) = Sij{^) (4.41) 
By supposition, > ^j since fj acted as the identity, so Si,j{^) = Si,j{^). If fj ~ s^j \ then 

1 = fi--- fj~iSjSj ■ --SiisiSi+i ■ ■ ■ s-j-is^ SjS.j-1 ■ ■ ■ Si+iSi{p,)) = s\j^{p) (4.42) 
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for some 

Applying these operations again and again we obtain that 

1 = Ii--- fN-iSN-i ■ ■ ■ si/i • • • ft~i{gi ■ --griv)) (4-43) 

where the y^'s are such as specified in the statement of the lemma, and where all the /^'s and /fe's act as 
Sfc. But replacing = = in the previous equation, we obtain that 

7 = 51... (7^(77) (4.44) 

which completes the proof of the lemma. □ 

Proposition 6. We have 

D* {u; q, t) mA = £a- (""; 9, t)mp,+ ^ va_q, (4.45) 

S1<A 

and, similarly, 

D® {u; q, t) niA = s^e, {u; q, t)mA+ ^ vau rriQ (4.46) 

f2<A 

Proof. Let to be a fixed integer. Define rj® to be equal to (77 + 1™)^, and define 77* to be equal to 77+. 
We say that a row i of 77® is fermionic if rj® ^ 77* and bosonic otherwise. 

We will first show that if appears in YiX^ then 7* < 77* and 7® < r\® . It is immediate from the 
definition of the Bruhat order on compositions and the fact that Yi acts triangularly on monomials that 
7* < 77*. We have left to show that 7® < 77®. From Lemma [SJ we only need to show that if 7 = gkij])-, 
for f^fc such as specified in the statement of the lemma, then 7® < 77®. Suppose that = s^j-. From 
LemmaEl have 77^ > 77^. The only non-trivial case is when i < m and j > m. In that case, it is easy to 
see that 7® will be obtained from 77® by interchanging a fermionic element and a bosonic one, with the 
fermionic one being the largest. This is easily seen to imply that 7® < rj®. Now suppose that gk — sfj. 
Again the only non-trivial case is when i < m and j > m. In that case, given that £ < {rji — rjj), 7® is 
obtained from 77® by modifying a fermionic element and a bosonic one in such a way that none of the two 
modified rows is larger than the largest of the original ones. It is then immediate that again 7® < rj® . 

Suppose now that A is a superpartition in the fermionic sector m (we will consider that A is also the 
composition {Af, . . . , A^, Af , . . . ). Then 



D*{u;q,t) TOA 



A™ A A*. 



N 

o-eSw w£S,n<: i=l 

Letting 77 = ii;(A) we have that 77* = A* and 77® — A®. From our prior analysis, we thus have that all 
the terms a;^ that appear in n^i(l + 7iFj)x"'(^) are such that 7* < A* and 7® < A®. The triangularity 
of the action of D*{u\ q, t) is then immediate. The triangularity of the action of D®{u; q, t) is proven in 
the same manner. Finally, proceeding as in the proof of Proposition |4l it can be checked that 

N 

JJ(1 -I- uYi) — £a* (u; q, t) + lower terms 
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and similarly, that 

m N 

= e\e> {u; q, t) + lower terms 

i—l i—m+l 

This completes the proof of the proposition. □ 

Proposition 7. The Macdonald superpolynomials are unitriangularly related to the monomials. In other 
words, 

Pa = TOA + ^ CAo (g, t) mn , (4.48) 

0<A 

where we observe that c\^{q,t) does not depend on N from Proposition\3i 

Proof. The triangularity is almost immediate from Propositions |4] and [G] Suppose that there exists a term 
rriQ such that i7 ^ A in Pa and suppose that Q is maximal among those superpartitions. Then by Propo- 
sition [S] the coefficient of in D*{u;q,t)P/^ and D® (u; q,t)P\ is respectively equal to CAn£n'(u;q,t) 
and CAQ£n@ {u; q, t). Since we cannot have eq. (u; q, t) = ea* {u; q, t) and Eq® (m; q, t) = Ea® 1i the 
same time {eA*{u; q,t) and eA®iu;q,t) uniquely determine A), we have the contradiction that Pa is not 
an eigenfuntion of D*(u;q,t) and D®{u;q,t) with eigenvalues eA*{u]q,t) and eA»{u;q,t) respectively. 

We now have to prove that the coefficient of ruA in Pa is equal to 1. To prove this, we follow [261 
Lemma 5.5]. First, observe that wla can be written as 

rriA = ^ J 01 • • • e^A„, ^"^''^ I ■ (4.49) 

We start with the expression (j4.ip for Pa, written compactly as 

Pa = CA ICa0l---9mAraU+.EAR (4.50) 

CTeSiv/(s„xs,„c) 

where the constant ca given in (|4.1I) (or in (|4.7I) ). It suffices to concentrate on the coefficient of the term 
in x^. ^From (|3.9p . we see that it can only arise from the dominant term 

Pa = CA Y ICa \0i--- e„Ani Y '^■^ ^^'^ + ieiuvs , (4.51) 
<TeSN/(s„xS,„c) y liiGS^c J 

where it should be observed that T„ acts on the variables a;^+ia;m+2^ ' ' ' ^n""^^ ■ Let rj — (Ai, . . . , A„i, Aat, . . . , Am+i) 
and write: 

Arnx^" = (-l)'"^(^")A„a;!^ (4.52) 
where inv(A") = m{m — l)/2. We thus have: 

Pa = (-l)(")cA Y ICa [Oi--- OmA„, Y + tenuis. (4.53) 

<Te5jv/(S,„xS„,c) y u)GS„c J 

It is easy to show (from the explicit action of Ti) that f26j Lemma 2.3]: 

T^a;^ = a;'"(^) + ^ d,x\ (4.54) 

where c?^ G Q(t). In the following, we denote by [x'^]F{x) the coefficient of x^ in the expression F{x). 
The coefficient of the term x^ in [/^c x— is given by 
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where £{w) is the length of the permutation w. Suppose that all the parts of A'* are distinct. Then there 
is only one permutation w that can give ■w{{A'')^) — A* and its length is given by £{w) — inv(A''). Now, 
when there are repeated parts in A*, inv(A'*) is the length of the permutation of minimal length such 
that w{{A^)^) = A^. However, we must also consider the contributions resulting from permuting these 
repeated parts. So, in general we can write: 

[x^] J2 r^a^^=i'""(^°'n ( J2 i'^'^'M • (4-56) 

where m{i) = n\s(i). Using X^o-eSt- ~ I^]*-' then obtain: 

K 

[x^] J2 T^^^ = t''"''^^'^^l[^^A4m- (4.57) 

Now, since U^cX^ is symmetric in the variables Xm+i, ■ ■ ■ ,xn and AmU^cX- is antisymmetric in the 
variables Xi, . . . , Xm, the monomial toa is reconstructed with these actions and multiplication by 0i • • • 0m- 
Hence, we have 

A? 

[mA] J2 )Ca{ei---e^AmU+.xn.) ^t'''-(^'^Y[[n^.{i)]tl (4.58) 

<TeSN/(S„xS„c) i=0 

which immediately gives 

[ruA] Pa = (-!)(") CAf"-(^°) Ht'^A^ (*)]*! = (-l)^") ca ^'""(^^^/a. (t) = 1. (4.59) 

1=0 

□ 

Corollary 8. The P\ 's form a basis of the space S^fq of symmetric polynomials in superspace. 

Recall that £)* (resp. D®) is the coefficient of u" in D*{u;q,t) (resp. D®{u;q,t)). 

Corollary 9. The 2N operators Z?* and Df for n,i = l,...,iV are mutually commuting when their 
action is restricted to . 

4.4. A simplified eigenfunction characterization. We end this section with a characterization of 
the Pa's as common eigenfuntions of two commuting operators. In the notation of Corollary [HI we have 



N 



From the linear term in u in (|4.17p . we see that the eigenvalue of the above operators on Pa are: 

D°PA = eAiPA, with oe{*,®}, and where e^^^ ^^q^H^-\ (4.62) 

i 

Given these two operators, it is natural to consider the following differences: 
= ^(Dl - Df) ^ ± E + + 
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and 
O2 



^^{qDt - Df) ^ f: 



From (j4.62p . we get 



and 



=OcreSiv/(S™xs,„c) 



OiPa 



O2PA 



9-1 



9-1 



N 



AT 



Pa 



Pa 



Observe that the two eigenvalues are in one-to-one correspondence with A. Defining 

N 



m=0(TeSiv/(S„xS„c) 

where K is the inverse of Yi : 



( A,n{x) 



(4.64) 



(4.65) 



(4.66) 



(4.67) 



Yi = ^Ti_i ■ ■ ■ TiwT/v-i ■ ■ - Ti, with Tj = T- ^ and ui — ui 



We have 



OiPa 



■ N 



1 ~ ^3 



Pa. 



(4.68) 



(4.69) 



1/9-1 

Finahy, we define Ei^n = d and £;2,jv = O2 - E!=i I* 

is easy to see that 

EimPk^{ E 9"'''^'"MPa and Ss.atPa = I ^ {q^' - l)t^-'\ Pa (4.70) 

\i:Af#A* / \i:Af=A* / 

have eigenvalues that do not depend on N . This property explains the substraction of X^iLi i^- the 
definition of £'2. at: it ensures that the eigenvalue does not depend upon the zeros in A^. This, and the 
fact that Pa is stable with respect to the number of variables, allows us to define 



and E2 — lini E- 



2,N 



El — limiJi^AT 
We have the following characterization of Pa- 

Proposition 10. The polynomial Pa is the unique polynomial in ^ such that 
(!) Pa = TOA + J2n<A "Af2 wn 

(2) EiPa = (E,:Af^A: q-^^'^t''^) Pa and i^2PA = (E,:Af= A* - 1)*'"') ^A 



(4.71) 



Proof. From Proposition [7] and (|4.70p . the superpolynomial Pa satisfies the two properties. Since Ei and 
E2 have together distinct eigenvalues, the two properties characterize Pa. □ 
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5. Orthogonality and existence 

5.1. Kernel and orthogonality. Let x = {xi,X2, ■ ■ ■) and y = (j/i, ?/2, ■ • ■ ) be two sets of commuting 
variables, and \el 6 — (6*1, 6*2, ... ) and = (0i, 02, ■ . • ) be two sets of anticommuting variables. We define 
the following reproducing kernel [7]: 

K(x, 0; 0) = n { (5.1) 

oo 

(a; 9)00 -n(i (5-2) 

K{xAy.4>)=K\x-y)\{(l+-^^ ) (5.3) 



where 



Observe that 



where 



K\x,y)=\{p^^^ (5.4) 

is the usual Macdonald reproducing kernel [23l eq. VI. 2. 4]. It is straightforward to show that 

Kix, e- 2/, 0) = ^(-l)(")zA(q, t)-^PA{x, 9) pA{y, 0), (5.5) 

A 

where z\{q,t) was defined in (|1.4I) . Recall from Theorem [T] that the factor ZA{q,t) is the norm of 

the scalar product of the power sums, i.e.: 

{{PA\pn))g.t = (-1)(") ZAiq,t)6An. (5.6) 

The following propositions are standard and can be proven using methods similar to those found in 
Macdonald's book 23 . 

Proposition 11. For each n,m, let {ua} and {va} be bases of^"'"^, where is the subspace of ^ 

of degree {n\m). Then the following conditions are equivalent: 

(1) {{uA\vn))q,t = Saq for all A,n 

(2) K{x,e;y,(p) ^Y.A'^J^i^^^)'^J^iy''l') 

Proposition 12. Let E : £i ^ £i be a Q{q,t)-linear operator. Then the following conditions are 
equivalent: 

(1) {{Ef I gh,t = ((/ \Eg)),^t for all f,ge^ 

(2) E'-'''''^K{x,e;y,(f>) = E'^y''t'^K{x,e;y,(f>), where E^^'^'> (resp. acts on the variables x and 
9 (resp. y and (p). 

The rest of this section will be devoted to the proof of the following proposition, whose corollary 
implies Theorem [TJ 

Theorem 13. We have that 

Ei'^^'^Kix, 9- y, 0) = E'f'^^K{x, 9; y, cj,) and Ei'^'^^Kix, 9; y, cj,) = ^^^^^if (x, 9; y, cj,) (5.7) 

Corollary 14. The Macdonald superpolynomial Pa is such that 

1) Pa = rnA + lower terms 

(5.8) 

2) {{PA\Pn))q,t = if A^n, 



MACDONALD POLYNOMIALS IN SUPERSPACE 



17 



Proof. The triangularity was proven in Proposition [T] By Proposition 1121 and Theorem [13] we have 

{{EiPa I Pn))qA = {{Pa \ EiPn))q,t and {{E2PA \ Pn))g,t = {{Pa \ -BaPo)),,* (5.9) 
Given that the two operators have together distinct eigenvalues, this immediately gives 

{{PA\Pn)),,t^O if A^n. (5.10) 

□ 

5.2. Self-adjointness of Ei and E2. The proof of Theorem [T51 is quite involved. It relies fundamentally 
on Proposition I16[ whose proof was kindly outlined to us by Alain Lascoux [21] . Theorem [T3] follows 
from the following proposition since Oi = Ei pj and O2 differs from £'2, at by a constant. 



Proposition 15. Let KN{x,9;y,(j)) be the restriction of K{x,9]y^(j)) to N variables. We have 

y, 

(5.11) 



oi^'^'^KNix, 9; y, cjy) = o'f^'^^KNix, 6; y, 0) and o'^2''^Kn{x, 9; y, cj,) ^ o'f-'^^KN{x, d; y, , 



where the superscripts indicate the variables on which the operators act. 

The first step in the proof of Proposition [15] will amount to reformulate the conditions (|5.1ip in a more 
tractable form (this is Proposition [T5I below) . 

Firstly, it is not difficult to see that the coefficient of 6*1 • • • 9m(j)i ■ ■ ■ (f>m in Kn{x, 6; y, (f) is equal to 
(up to a sign and a power of q): 

\a OA M } ir I a A\ K%{x;y)Ajn{x)Ajn{y) 

Wi ■ ■ ■ Om(t>i ■ ■ ■ (Pnil KN{x,9;y,(l)) (X _ _^ -, (5.12) 

where we recall that [^i • • • ^m^i • • • (t>m\ f stands for the coefficient of • • • 9jn4>i ' ' • 0m in /• This is seen 
as follows: the coefficient of 6*1 • • • 9m(j)i ■ ■ ■ (f>m in Hi j [1 + ^i</'j/(l ~ ^iVj)] is (up to a sign): 

J2 i-if"'^ frwr^ V = E(-i)'^'^' E ^"2^"*'" = E(-i)^''"'''^^'' v-M") iy)^iy)^ (s-is) 

a=i(i-^.2/.w) . , , 

where 77 is a composition with distinct parts (otherwise the result is zero by antisymmetry), 77+ is the 
partition corresponding to 77 and sign(77) is the sign of the permutation that changes 77 to 77+. The 
second equality is obtained by interchanging the two summations and using the expression of the Schur 
polynomial as a ratio of two determinants (cf. |231 eq. (3.1)]). Next, by splitting the sum over 77 into 
a sum over 77+ and a summation over permutations of 77+, we can rewrite the last expression as (with 
77+ -(5(™) ^ A): 

E (-i)^^-' n- a'xvr.) = ^'"(-)^™(^)E^^(-)^>(^) = . (5.14) 

using j23[ eq. (4.3)] in the last step. By substituting Xi — > Xi/q and multiplying the result by K^{x]y), 
we recover (|5.12p . 

More generally, the coefficient of • • • 9m(f)ji ■ ■ ■ 4>j,ri (with ji < ■ ■ ■ < jm) in Kn{x, O: y, (p) is 

\a OA A \ V i a A.\ K-iv) K-iy) K%{x;y)A„,{x)Amiy) 

[9i--- 9^4>j, ■ ■ ■ (t>jj Kn[x, 0; 27, 0) oc Kl'^f^ ■ ■ ■ K'^j^— _ _^ -. (5.15) 

lll<ij<mU 1 ^iVj) 

Given this result, the coefficient of 9i-^ ■ ■ ■ 9i^^(j)j-^ ■ ■ ■ in 0^-y'^'^KAi{x, 9; y, (j)) is proportional to 



.(X) j.(x) ^{y) j^{y) A^(x) .7(r.)^ ^^jx; y)Ai^{x)A^{y) 

^M. • • • i^m.„i^l.. • • • i^m.„ ^^(^) (^-1 + ■ • ■ + ) ni<,,<„(l - q'^X^y,) ' 
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Similarly, the coefficient of Oi-^^ ■ ■ ■ Oi^ipj-^ ■ ■ ■ (jjj^ in O^^ ''^^ K {x , 0; y, (j)) is proportional to (with the same 
proportionality factor as above): 

The first relation in (|5.1ip is equivalent to the equality of both coefficients. Canceling the permutation 
operators in the equality between (|5.16p and (|5.17p yields 

^m{x) -^(x) y(^). K'^N{x;y)Al^{x)A^{y) ^ Am(y) .>(y) y>fa)N K%{x;y)A„^{x)Al^{y) 

^Ini^y ' 'Ui<.,<rna-<i-'^^y,) ^LivV' ^ Dk^.x™ ( 1 " ^^.y. ) ' 

(5.18) 

Proceeding similarly for O2, and using commutativity of the type Yi^^ f{y) = fiy)Yi'^\ we obtain the 
two relations appearing in the following proposition, whose proof thus implies Proposition [151 

Proposition 16. We have 

iii<i,j<T?xv^ y ■^lyj) i\.i<i,i<m\^ y xiyj) 



an 



d 



fv(x) , , ^^ix), K%{x;y)Al^{x)Al^{y) _ , K%{x;y)Al^{x)AlM 

for all m = 0, . . . , N , where and act respectively on the x and y variables. 

The proof of the proposition relies crucially on Lemma 1171 which is formulated in terms of divided 
differences d, Sec. 7.1] 

d,^- i r(l-s.). (5.21) 

(Xi - Xi+l ) 

The divided differences obey the braid relations [20l Sec. 7.3] 

di+ididi+i = didi+idi, didj = d^d^ if li-jl>l, (5.22) 

and the nilpotent condition = '20', eq. 7.1.5]. As is the case for the T^'s, if Si^ ■■ - Si^ is a reduced 
decomposition of w then d^^ stands for . . . di^. 

The following lemma was stated by Alain Lascoux [21j. We provide our own proof of it. 

Lemma 17. Let d^,-^ be the divided difference associated to the longest permutations of Sm ,2(7| Sect. 
7.6] . Then the following identity holds: 

where the superscript indicates that the divided differences act on the y variables. 

Proof. We first observe that IlKi j<m(l ~ ^iyj)^ being fully symmetric in the j/j's, vanishes when acted 
on by Si^. Therefore, multiplying both sides of the identity (I5.23P by Hk^ j<m(l ~ -21%), we can then 
use Leibniz identity [IHl eq. (7.1.10)] to commute the product with the divided differences. We then note 
that the product can be factorized as follows: 

n (1-^*%)= n n (5-24) 

l<2j' <m -i+j <m4-l i+j>m+l 
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Hence, the identity (|5.23|) is equivalent to 

/ 



i-\-j<m i+_7>m+l 



d[^lQ(z,y). (5.25) 



It is well known [201 Sect. 7.6] that 

d^^livl' ■■■V^n )^^ (5.26) 
unless all the a^'s are distinct. Observe that after expanding the products on the rhs of (|5.25p . the 
appearing monomials • • ■ y°j" will all be such that ai, . . . , am < m — 1. Indeed, the power of yj equals 
the number of distinct factors Zi that can appear in the coefficient of j/J^ and the maximal value of this 
number is, with j fixed: 

#{i \ i + j <m} + \i+j>m + l} = m-l. (5.27) 

The only option to have distinct a^s is thus for (ai, . . . ,a„i) to be a rearrangement of (m — l,m — 
2, . . . , 1, 0). Therefore, the rhs of (|5.25p is a polynomial in j/i, . . . , of degree 0, that is, the result does 
not depend on the variables y. We now show that it is i-antisymmetric in the z variables, i.e., that 

Ti'^dLtQi^, y) = -d[^lQ{^. y) (5-28) 

for all k. We have that = wsm-k for some permutation w G Sm- Hence, it suffices to prove that 

Tt^tl,Q{^,y) = -d^^l^Q{z,y). (5.29) 

It is easy to see that Q{z,y) is symmetric in both y,n-k,ym-k+i and Zk,Zk+i except for the factors 
(1 - tzky„i-k){i - Zk+iym-k+i)- A direct calculation yields 

^m-fc(l ^ tZkyrn-k)ii - ^fc+i y,„_fe+i ) = -{tZk - Z/c+l), (5.30) 

from which (|5.29p follows immediately since Tk{tzk — Zk+i) — —{tzk — Zk+i). Finally, the rhs of (|5.25l) is 
a polynomial in z of total degree m(m — l)/2. Since the only such ^-antisymmetric polynomial is A^j(z), 
(|5.25p holds up to a constant. The coefficients of • • • on the LHS of (j5.25p is clearly 

(— t)^^). On the rhs of (|5.25p . the coefficient of • • • Zm-i is du:^ acting on a certain polynomial 

p{y) in y. In p{y), the only monomial whose exponent is a permutation of (m — 1, m — 2, . . . , 1, 0) is 
yT~^yT~'^ ■ ■ ■ ym-i- Given that its coefficient is {—t)^'^ \ the result follows. 

□ 

We now turn to the proof of Proposition [T51 

Proof. We will assume throughout the proof that N > m. The case N — m can be easily obtained 
as a simplified version the case N > m. Note that in the following arguments we will never worry 
about constants depending on q and t (which are irrelevant to the symmetry). For instance, we write 
f/m '■^^ A^(a;) oc A^(x), meaning that the two expressions only differ by a constant. (Recall that U,^ = 

We first prove equation (|5.19p . This amounts to show that 

Fi., y) = [Yt^ + + y(^)) ^^(-^^)f^(-)5(^) (5.31) 

is symmetric in x and y, that is, F{x, y) = F{y, x). Since A*„(x) oc Um ^^''A^(x) and because Um ''^'^ and 
A^(?/) commute with the f/'^^'s, we have 

F(x, y) cc AL(y)[/,; (^\yI^^ + ■■■ + yJ.)) A^(x) JC°,(.; y) ^^^^^^ 

i-h<i,j<m\^ 1 ^'■Vj) 
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Recall the expression for the inverse Cherednik operator in (|4.68p and that for Tj in p.6|) . Since 
T-^^Al^{x) = —t~^Al^{x) and Um'^^^Tj^^^ = —Um''^^ whenever i < m we have that, up to an irrele- 
vant i-power, [/„ ^''^y}''^ Al^{x) can be replaced in (lO^ by ^'''>uj^'''>fj^l^ ■ ■ ■ T^'>AlJx). Hence 



Using (see p 
which gives 



Sect. 7.6]) a^fj = [A„.(a;)]-iA,„ and ([XTC]) . we have 



Using the identity (|5.23p with Xi — qzi, we obtain 

Fix, y) ex AL(y) AL(x)9fa29(->(-)4-2, ■ ■ ■ f^) n.+,<„.(l "^g-^^-^.y.) 
To prove the symmetry of F{x,y), it thus suffices to prove the symmetry of 

The only part of the term in parenthesis that depends on the variables Xm, ■ ■ ■ ,xn is (1 — q^^x„iyi)^^ ■ 
We have 



(5.33) 

(5.34) 
(5.35) 

(5.36) 
(5.37) 



(x) 



N-1 



. . . y(^) 



1 



{I -q ^Xmyi) 



t 



-N+m 



which implies that 



vn,+,<„+i(i-«-'^.!/i)y vn,.„+i(i-9-'«.i'i). 

We then straightforwardly compute 



K%{x;y). 



io^^^K%{x;y)=K%{x;y) 



(5.38) 



(5.39) 



(5.40) 



n (1-^9 ^Xiyj)= {l-tq ^Xiyj)\ ]J (1 - tg ^Xm+i-jy,) 



i-\-j<rn 



i-\-j<ni 



J=2 



and 



(x) 



Yl (l-q ^Xiyj)= Jl {l~q ^Xiyj)\W{l-q ^Xm+2-jyj) 



z+j <m+l 



i-^j<.'m-\-l 



{x) 



n,=m+i(l-*'? ^Xiyi) 



n.L„+2(i-9 ^^^yi) 

All of the terms in the rhs of these expressions are symmetric in x,y except those in square brackets. 
Multiplying the terms in square brackets, the symmetry of G(x, y) then depends on the symmetry of 



(1 - tq ^x,nyi) 

UT=ii^-t<l~^xiyj) 

(5.41) 

(1 - q-^x„i+iyi) 
njli(i - q^^xiyj)^ 

(5.42) 
(5.43) 



{l-q ^xiyi) ' 



{l-tq ^Xmyi) 



n" i(i - i-'^iyA / nf=™+i(i - tq-'^^yi) 



n 



N 

i=m+2 



{l-q '^x^yi) 



(5.44) 
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But the previous expression is equal to 



(5.45) 



which is obviously symmetric in x,y. This proves the symmetry of G{x,y) and therefore equality (j5.19p 
holds. 



We now prove eq. (|5.20p . proceeding as in the proof of (|5.19p . This amounts to proving that 

^(^) _i_ _i_ \^(^)\ 

satisfies H{x,y) = H{y,x). We have 

Hix,y) « AL(2/)L/„(-)C/+^H>lti + • ■ • + if^) ^ ) (5.47) 

Since U^a — tUj^c whenever i > m + 1 (and similarly for on any function symmetric in 
Xi, Xi+i), we have 

ff(x,2;) (X AL(y)L^„(^)C/j;i^^^(^)T(^)---Tif):p^ ' (5-48) 

Using (|5.34p . we have thus 

Using the identity (15. 23^ with Xi — qzi, we obtain 

Hix,y)o.AUy)AU-)diyldj,^lu:^J^^^^^^ (5.50) 

llj+j<m+lU 1 ^iVj) 

To prove the symmetry of H(x,y), it thus suffices to prove the symmetry of 

Lix, y) = C/:i^)a;(^)T(^) ■ • • f^^^ f ' '^"'"'1 <(x; y). (5.51) 

The only part of the product that depends on the variables Xm,Xm+i is is (1 — q~^Xmyi)~^ . We have 

fM 1 ^ (l-tg-^x„yi) ^ ^^^^2) 

" - q'^x^yi) {I ~ q-^x,n+iyi) (l - q'^x^yi) 

Adding the previous expression to the product in the rhs of (I5.5ip . the only factor that is not symmetric 
in Xm-i,Xm is (1 — q^^x„i-iy2)^^ ■ We have this time 

1 ^ t^' {l-tq-^x„r-iy2) ^ 

(1 - <7"^a;„_iy2) - q^^Xmy2) ~ q^^Xm~iy2) ' 

Continuing in this manner we get 

fix) rfix) ( Il:+j<mi^-tq~^x,yj) \ ^ f Y{,+,<m{^-tq-^x^y,) \ ( UZii^ ~ tq-^ x„,+i^,y,) 

(5.54) 

which implies that 

L(., y) oc C/+i^)c.(^) /' n.+,-<.„+i(l -^g-^^»y.) \ _ y). (5.55) 

V n..+j<™+2(l-9 a:,y,) / 
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The following actions of w*-^^ are straightforward 



u:^^^K%{x;y)=K%{x-y) 



]\f=l{^-XNVi) 



UJ 



n (i-i?"'^ 



^^yj) = Yi i^-tq ^x^y^) 

i-\-3<rn 



]^(l - txNyj 



^v^. n (^^'^ ^x,yj)^ Y\ ^x,yj) 

i+j<m+2 i+j<m+l 



J = l 
m+1 



]J (1 - XNyj) 



and 



The product of the terms in square brackets of these expressions gives 



n 



Hence 



L{x,y) oc 



i-\-j<m 



(1 - tq-^Xiyj) 



[(1 - XNym+l)] 



n 



i—m-\- 



n 



TV 



n 



n 



N 
i—rn-\- 



i(l - XNyi) 



All the terms in this expression are symmetric in x, y, except possibly 



n 



AT 

.i—m-\- 



i(l - XNyi) 



But the symmetry of this last expression follows from the well known symmetry of the m — case (the 
usual Macdonald case). Therefore L{x,y) is symmetric and (|5.20p holds. □ 

5.3. Duality. Wc end this section by generalizing to superpace the standard duality property that re- 
lates the Macdonald symmetric functions P\{q,t) and Py{t,q) [23l Section VI. 5]. Our method relies 
on the orthogonality and triangularity of both the Macdonald superpolynomials P{^{q,t) and the Jack 
superpolynomials P["' , respectively established in Corollary [T3] above and Theorem 1 of [T2] . We also 

exploit the duality between the Jack superpolynomials P^"'' and P^? ' given in Theorem 27 of [12]. Note 
that in what follows, only the special case a = 1 is relevant. 

The algebra of symmetric functions in superspace is naturally equipped with two homomorphisms, 
the first of which being 



(5.56) 
(5.57) 
(5.58) 
(5.59) 
(5.60) 

(5.61) 
(5.62) 



i^q^tPr = (-1)'"^ 



1 



l-V 



■Pr 



(5.63) 



This is an extension to superspace of the standard homomorphism defined in L23j Section VI. 2]. Second, 
we introduce the following homomorphism that affects only the fermionic power-sums: 



l^qPr = Pr ^qPr ^ Q Pr ■ 

Combining the two homomorphisms, we get 

^qA '■= ^q O ^q,t , 

which is such that 



1 



where 



,,^^(-l)IM-^(A=). 



■PA, 



(5.64) 
(5.65) 

(5.66) 
(5.67) 
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When q = t = 1, flq^t reduces to the homomorphism uj of (llj . whose action can be summarized as foUows: 

ujpA^ujAPA- (5.68) 

Equations (|5.66|) and (|5.68p immediately imply that, for all f,g & ^ (and by linearity, it suffices to verify 
the case where f = pa and g = pn), 

{{n-}f\g)),^t^{{ujf\g)), (5.69) 

where the scalar product (( | )) on the right-hand side is defined as (( | ))g,t in (|1.4|) . but with q = t = 1. 
Note that both Qq^t and lo are in fact automorphisms of Their respective inverse are: 

Q~l — flt.q ° ^{qt)-^ and (jj^^=u}. (5.70) 
In each homogeneous component .^"'™ of we also have 

r!-j = (fV")f^t-i,g-. (5.71) 

Now let sa be the Schur superpolynomial associated to the Jack superpolynomial P["^ [T^], which 
means sa := Pa'' ■ We stress that sa is not equal to the Schur function sa defined later in the article as 
the q,t-)-0 limit of PA{q,t) (cf. eq. (H^))). Then according to Theorem 1 of [H], 

((saIsj,)) =bX'<5A,n. (5.72) 
Moreover, from IIT, Theorem 27] applied to a = 1 (and recall that A' is the conjugate of A), 

wsA = Sa- = b^^^sA' (5.73) 

The expression for the normalization constant b^^ is known, being equal to (-1)(") ||P|^^ f, where 
||p["''|P is given by Eq. (18) of [13j. When specialized to a = 1, this normalization factor reads 

bA = (-!)« n °^^r;"tl - (^-^4) 

W OA* + *^A® + 1 
A 

Since upon conjugation, the role of the arm-length and leg- length (defined in p.3p ) are exchanged, it 
satisfies 

V =bA': (5-75) 

so that we can write 

SA:=bASA and ((sa | Sjj)) = (5a,o. (5.76) 

Theorem 18. Let Qa — hA{q,t)PA{q,t) , where hA{q,t) ~ {{PA{q,t) \ PA{q,t)))q l . Then, the following 
duality holds: 

nq^tPA{q,t) = {qt-^)^^^QA'{t'\q-^). (5.77) 

Proof. We proceed essentially as in [23} Section VI. 5]. Note that in what follows, we assume that all 
polynomials are homogeneous and belong to ^"'™, which is finite dimensional. 

Thanks to the orthogonality of the Macdonald superpolynomials established in Corollary [T3] and the 
inversion formula (I5.7ip . the stated duality property is equivalent to 

{{n-}PA'it-\q-')\Pn{q,t)))q^t = <5a,o, (5.78) 

By virtue of (j5.69p and iv^^ — tu, the last equation turns out to be equal to 

{{LoPA'it-\q-^)\Pn{q,t))) = 5as^. (5.79) 

Now, let A{q,t) be the transition matrix between the PA{q,tys and the sji's, that is 

PA(<?,i) =^AAo(g,Osn. (5.80) 
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Let also J be the involutive matrix with elements Jao — Sa'.q- Then (|5.79|) is equivalent to the following 
matrix equation: 

JA{t-^,q-^)JA{q,ty = I, (5.81) 

where we stress that A{q,ty is the transpose of the matrix A{q,t). This is the equation we will prove. 
Before attacking this problem directly, we need to derive a number of auxiliary results, to which we now 
turn. 

Let p, s, and s* be the column vectors with the A-element equal to pA, sa, and respectively. Let 
b, and C(g, t) be diagonal matrices whose non-zero elements are respectively given by bA (defined in 
(|5.74p ). Ca = {{PA I pa)) and CA('7,i) = {{PA \ PA))q,t, so that 



£(A= 



Ca = i-lP^A^ and Uiq^t) - (-1)(")za= g'^"' H JZ^' (^.82) 

i—l 

We also define X as the matrix with entries Xau = {{pA Then by making use of the orthogonality 

with respect to (( | )) of both the sa's and the pa's, one readily obtains 

p^Xs, s*^X'C^P- (5.83) 

Together with s* = bs, these imply 

X' = bX-^C- (5.84) 
Moreover, let U{q, t) and be the matrix with elements 

UAn = ((sA|s^)),,t. (5.85) 

The use of the previous two equations then leads to 

XU{q,t)X-' ^C\iq,t), (5.86) 

whose inverse version reads 

XUiq,t)-'X-^ = aq,tr\. (5.87) 
From the explicit expressions of C{Q,t) and ( in (I5.82p . we see that 

C(9, t)-\ = {eq--)C'at-\q-'). (5.88) 

Because X is independent of q and t, the relation (|5.87p implies that 

U{q,t)-' ^{eq-^^)U{t-\q-'). (5.89) 

Furthermore, we have, using (|5.73|) . the adjoint character of lo and = 1, we have 

{JU{q,t)J)An ((sA' I slv))q,t = ((ws^ | i^sn))q,t = {{s*a \ so))g,t, (5.90) 

so that (translating the first and third equality above) 

JU{q, t)J = C/(g, ty ^ b U{q, t)^-^ . (5.91) 

One last identity concerning U {q, t) is needed: 

A(g, t)h-'U{q, tyA(q, ty = b(q, t)-\ (5.92) 

where b(g,i)^^ is the diagonal matrix with entries b^^(q,t) — {{PA{q,t)\PA{q,t)))q^f The relation (|5.92p 
follows directly from the orthogonality of the Macdonald superpolynomials and the definition of A{q,t). 

We are now in position to prove (|5.8ip . From the triangular expansions of PA{q, t) and sa (which are 
both of the form ttia + lower terms), we know that the matrix A[q, t) is strictly upper unitriangular. We 
recall (see for instance (23 I- (6-2)]) that a matrix M is strictly upper (uni)triangular if and only if JMJ 
is strictly lower (uni)triangular. Hence, the matrix 

B = JA{t-\q-^)JA{q, ty (5.93) 



A? 
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is strictly lower unitriangular. Therefore, in order to prove that B is the identity (which will prove ()5.8ip ). 
it only remains to prove that B is also upper triangular. The use of the second equality of ()5.92|) and 
(I5.91|) . under the form t/((7,i)b^^ J ~ 'b^^JU{q,t), allows us to write 

h{q, t)-^B-^ = A{q, t)h~^JU{q, t)A{r^ ,q~^Y^ J (5.94) 

Now, by isolating U from ea. (|5.92p and inverting the result by using eq. (|5.89p , we get 

\^{q,t)-^B-^ = {t-"q")Aiq,t)h-^ Jh-^ A{t-\q-^y h{t-\q-^)J. (5.95) 

However, by exploiting (|5.75l) and the definition of J, one readily shows that h^^Jh^^ = J. Thus, 

Hq,t)-'B-' = {t-\")A{q,t)JA{t-\q-'yb{t-\q-')J, (5.96) 

The comparison with (|5.93p yields 

b(g, t)-^B-^ = (t-"q") B' Jh{t-\q-^)J. (5.97) 

bmce bothb(t-\q-i) and J\>{t-'^ ,q-^)J are diagonal matrices, and since B is strictly lower unitriangular, 
the last equation implies that B^^ is also upper triangular. Consequently, B is the identity matrix, which 
completes the proof of (|5.8ip . □ 

6. Another scalar product 

The value of the norm ((-PaI-Pa))^,* of the Macdonald polynomials in superspace was conjectured in [7]. 
This conjecture is reproduced below. We will now define another scalar product with respect to which 
the Macdonald polynomials in superspace are also orthogonal. This other scalar product is not bilinear 
anymore (it is sesquilinear) . Nevertheless, we will show that, remarkably, the norm of the Macdonald 
polynomials in superspace with respect to that other scalar product is (up to a power of q) identical to 
the conjectural expression for {{PA\PA))q,t- 

6.1. The conjectured norm of the Macdonald superpolynomials. We first present the conjectural 
expression for ((-PaIPa)}?,* given in [7,. It involves the quantities (recall that arm and leg- lengths were 
defined in (ESI) 

^^1= n (1 and 4= n (1 (6.1) 

s6B(A) seB{A) 

where B{A) denotes the set of squares in the diagram of A that do not appear at the same time in a row 
containing a circle and in a column containing a circle (this excludes for instance the squares (1, 1), (1, 2) 
and (3, 1) of A whose diagram is found in (|2.4I) ). 

Conjecture 19. The norm of the superpolynomial Pa defined in Theorem\^is 

((PA|PA)),, = (-i)(")gl^"lj^. (6.2) 

"•A 

6.2. The constant-term scalar product of the non-symmetric Macdonald polynomials. Let 

C.T.(/) denote the constant term of the Laurent series of / in the variables xi, . . . ,xn ■ Define the 
following scalar product on '^{q,t)[xi, . . . ,a;jv]: 

(/, 9)N,q,t ■■= C.T. {fix; g, t) g{x-^;q-\t-^) W{x; q, t)] (6.3) 

where 

W{x]q,t)= ]J (a:»/a:j;g)oo {qxj/xi]q)oo ^^^^^ 
i<i<j<N (^■''^/'^i' '^)°° {qtXj/xi,q)oo 
Note that this scalar product is sesquilinear, that is, for c — c{q,t) G Q{q,t), we have 

{cf,9)N,q,t^c{f,g)N,q,t and {f,cg)N,q,t = c{f,g)N,q,t (6.5) 
where c — c{l/q, 1/t). 
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Proposition 20. [23j[24] The non- symmetric Macdonald polynomials Eri{x;q,t) form an orthogonal set 
with respect to •)N,q,t- 



(6.6) 



The norm is explicitely given by tlie expression (cf. [26, Prop. 3.4] and [8l|3]) 
where, for a composition 7, 

^ = ^9"'^'^*^"''^'^] ^Yl[l-q''^'h^^'^+^], (6.7) 

sG7 sS7 

(we added two expressions to be needed shortly, and hj). The arm- and leg-(co)lengths in these 
expressions are given, for s = (i,j), by 

a{s) = 7» - J 

l{s) = #{fc - 1, i-l\j <Jk + ^<l^} + #{k = ^ + l, iV I J < 7fc < 7»} 
a'(,s)=j-l 

r(s) = #{fc = l,...,i-l|7fc >7,} + #{fc = ^ + l,...,iV|7fc >7,}. (6.8) 

6.3. Another orthogonality relation for the Macdonald superpolynomials. Let / £ be a 

symmetric superpolynomial of fcrmionic degree m. We define 

^.^^_(™)A^x) 

Am(x) 

where we recall that, as in Section HI [9i ■ ■ ■ 0„i]f stands for the coefficient of • ■ • dm in /■ Note that 
/* € Q(9) t)[xi, ■ ■ ■ , xn] since [9i • ■ • 9m] f is antisymmetric in the variables xi, . . . , Xm and thus divisible 

by Am{x). If / does not have a specific fermionic degree, it can be decomposed as / = /o + /i H ^ fr, 

where fm is the part of / of fermionic degree m. We then let 

r = /o' + /; + ■•■ + /;• (e.io) 

Definition 21. Let f and g be superpolynomials in I%n- We define the following sesquilinear scalar 
product on M: 

{f,9)f,q,t=T.U^(f^^9m)N,q,t (6.11) 

rn ^ 

The following proposition states that the Macdonald superpolynomials are also orthogonal with respect 
to this new scalar product. Moreover, it relates the norm of Pa in the two scalar products. 

Proposition 22. We have 

(^A,Po)f,,,t = HA^n. (6.12) 

Furthermore, 

JV^oo {l,l)N,q,t h\ 

? 

where m is the fermionic degree of A, and where = means that the equality is only conjectural. 
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Proof. By definition (Pa, Pn)^ g t = ^ have different fermionic degrees. We can tlius suppose 

tliat A and 51 liave fermionic degree m. The proof now follows the argument of jS^ up to eq. (j6.22p . Using 
expression ()6.19p . we get 

{PA,Pn)§,,, = ^AWcor^) J2 (UmUr^^E^^A-t)-'^^^T^T^'En^)N,,,t (6.14) 

= 'J^^^^ Y: E ((-0^<'^)r-:^.r-\^™C/,ti?A«,£;n«)^.,,, (6.15) 

where we used the fact that T^_\ is the adjoint of To- with respect to (•, ■)N.q.t- Note that Tg- and T^.' 
commute because they act on disjoint sets of variables. We then use 

i;±[/+ = t^U+ and T;±[/- = (6.16) 

in order to write 

{PA,Pn)'^,,,t = [iV-m]4-i!cn(t-i)(5(A.,A.),i?n-.)jv,,,t, (6.17) 
where we used X^o-gs ^^^'^^ — ['"]*•' ^^^^ where 

'S'(A»,A=) = CA?/;;;f^m--^((A<')«,(A=)«)' (6.18) 

is the Macdonald polynomial with mixed symmetry (also considered in [3] but only in the case where 
A = J^™) = (m — 1, . . . ,0)). The analogue of [1] Corr. 1], which is obtained as shown there using the 
generalization of lemma 2.5 of |26], reads: 

- l^i-t) '-T, T -— ^K(a«),<t(^«))- (6.19) 

By the orthogonality of the non-symmetric Macdonald polynomials, the rhs of (|6.17p is non-vanishing 
only when A = il , and a and a' are equal to the identity in (|6.19p . Hence 

/P P\« . . [N ~ m],-. ! A--)rf((A°)^,(A--)^) 

{PA,Pn)N.q.t = OAO (-1)^2^ 7 TTTT^ -J, 1 (-C-A"- , ^A« ) W,9,t , (6.20) 

jA^[t } a((A")«,(A=)«)^(A",(A=)«) 

which implies that {PA,Pn)N.q.t = if A 7^ 11. 

Furthermore, with the identities [26l eqs (5.16) and (5.15)]: 

[N-m]t-i\ . ^ [N~m]tl ^ &A^rf(A^)H , . 

/A=(i-1) /A=(t) /lA»e(A=)H' ^ ^ 

and using the expression (|6.6p for the norm of _Ba« , we obtain 

(l,l)Ar^5^f /lA=e(As)« d(A»,(A=)«) e'((Aa)H^(A=)«) 

In order to relate this expression to the conjectural expression for {{PA\PA))q,t, we need to consider 
the limit — 00 of ()6.22p and recall that |t| < 1, so that all the factors e, e' and b reduce to 1. The 
proposition will follow after establishing that 

hm = (_i)(?) li,, ^((A°)-A-)^(A^)" ^ i-lpA (6.23) 

N^co {l,l)N,q,t N^oo c((Aa (A=)n)AlA'- h\ 

The last equality is quite long to prove. The details are thus relegated to the appendix. □ 
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7. MACDONALD superpolynomials and generalized (g,i)-KOSTKA COEFFICIENTS 

In this section, we review the super-extension of the Macdonald positivity conjecture, exhibit sym- 
metries of the generahzed (q, i)-Kostka coefficients and present two new conjectures related to these 
coefficients. 

7.1. Generalized (g, i)-Kostka coefRcients. The generalized (g, t)-Kostka coefficients are defined from 
the integral form of the Macdonald superpolynomials: 

Ja = 4^a (7-1) 

(where /ij^ is defined in (|6.1|) ). It was conjectured in |7^ that the coefficients in the monomial expansion 
of Ja are polynomials in q and t with integer coefficients. 

We next introduce the Schur superpolynomials [7] 

SAix,0)^ PAix,0;O,O), (7.2) 

and their deformation 

SAix,0;t)^ ip{sA{x,e)), (7.3) 

where if stands for the endomorphism of Q{q, t)[pi,p2,P3, ■ ■ ■ ;po,pi,p2, ■ ■ ■] defined by its action on the 
power-sums as 

(p{Pn) = (1 - i")Pn and (fiPn) = Pn- (7.4) 

Remark 23. As was commented in [7], the existence of the limiting case sa{x, 0) = Pa{x, 0; 0, 0) does not 
follow from the existence of a solution to conditions p.ip since the scalar product is degenerate when 
q = t = 0. A better approach is to consider the limit in Definition 14.11 In this case, it is known that 
a non-symmetric Macdonald polynomial becomes a Key polynomial (or Demazure character) |16[ I22j . 
Unfortunately, because of the quantity l/i'"^(^'\ it is not immediate that the limit exists. To overcome 
this problem, we can define the Macdonald superpolynomials by (anti)symmetrizing Ea instead of Ear- 
This will be considered in a forthcoming publication. 

Given this, we have formulated in 7j the following striking version of the Macdonald positivity con- 
jecture. 

Conjecture 24. The coefficients KiiA{ci,t) in the expansion of the integral form of the Macdonald su- 
perpolynomials 

Ja{x, 9: q,t) = Y, KuAiq, t) Sn{x, 6; t) (7.5) 

are polynomials in q and t with nonnegative integer coefficients. 

The following symmetries have been observed: 

KnA{q,t)=K^,A'{t,q), (7.6) 

(cf. [21 eq. VI (8.15)]) and 

KnA{q.t)^q'^('''h^(^^K^,A{q-\t-^). (7.7) 
In the previous equation we used 

n(A) = n{SK) - d{A) with 71(A) = ^(i - 1)A, , (7.8) 

i 

where SA is the skew diagram SA = A®/(5("+i\ with the staircase partition (m-l,TO-2, . . . , 1,0), 
and where d(A) is defined as follows: fill each square s G BA (defined immediately after (I6.ip ) by the 
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number corresponding to the number of squares above s which are both in a fermionic row and a fermionic 
column (i.e., both ending with a circle); d(A) is then the sum of these entries. For example: 



d((2,l,0;l,l,l)) = 6: 








2 




2 




2 





d((2,l,0;2,l)) 










1 


1 










2 







d((2,l,0;3)) =0 

























(7.9) 

(In a sense, c^a is dual to the quantity (a introduced in [7] to describe the specialization of Pa). The 
quantity dA vanishes for m < 1, so that that expression [23l eq. VI (8.14)] is recovered when m = 0. 
Examples of {q, t)-Kostka coefficients are given in Tables [T] to [71 (Note that in the tables presented in [7], 
the transpose symbol (prime) is missing. These are Tables [T] - [S] below) 



An example illustrating the first symmetry property (j7.6l) is (cf. Table [T]) 

^(2,0:1,1) (2,0:2) (<?, t) = q + qt + q^t = -ftr(3,0;l) (2,0:2) {t, q), 



(7.10) 



since (2, 0; 1, 1)' = (3, 0; 1) and (2, 0; 2) is self-conjugate. An example of the relation (|7.7p is 

^(2,0;l,l) (1,0;2,1)('Z, t) = t + qt' + qt^ = qt^'K^^^^a) (i,0;2,l) (<?"', ^"') = qt\t-^ + + q-'t-^)- (7.11) 

The factor can be checked as follows (the diagram of A is filled with numbers that add up to d{A) while 
the skew diagram iSA, obtained from A® by dropping the squares marked by a x , is filled by numbers 
that add up to n{SA)): 



A=(l,0;2,l): 
Similarly, for q factor, we have 







1 






SA 



the power of t is 6 — 1 = 5. 



(7.12) 



A' = (3,l;0): 



SA' 



X 


X 








X 


1 





the power of gr is 1 — 0=1. 



(7.13) 



TABLE 1. KnA{q,ty for degree (1|1). 







(1;) 


(0;i) 






(1;) 


1 


q 






(0;i) 


t 


1 




Table 2. KaA{q,t)' for degree (2|1). 




(2;) 


(0;2) 


(i;i) 


(0;1,1) 


(2;) 


1 


q' 


q 




(0;2) 


t 


1 


qt 


q 


(i;i) 


t 


qt 


1 


q 


(0;i,l) 




t 




1 
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Table 3. Knp,{q,ty for degree (2|2). 





(2,0;) 


(1,0; 1) 


(2,0; ) 


1 


q 


(1,0;1) 


t 


1 



Table 4. KnK{q,t)' for degree (3|1). 





(3;) 


(0;3) 


(2;i) 


(i;2) 


(0;2,1) 


(i;i,i) 


(0; 1,1,1) 


(3;) 


1 




q + q^ 


q^ + 


q* + q^ 


q' 


q' 


(0;3) 


t 


1 


qt + qH 


q + q'^t 


q + q' 


q^t 


q' 


(2;i) 


t 


qh 


l + qt 


q + q'H 


q^ + q-^t 


q 


q' 


(i;2) 




qt 


t + qt^ 


1 + q'H'^ 


q + q^t 


qt 


q' 


(0;2,1) 




t 


t^ + qt-^ 


t + qt^ 


l + qt 


qt^ 


q 


(i;i,i) 




qt-^ 


t + t^ 


t + qt^ 


qt + qi^ 


1 


q 


(0; 1,1,1) 






t* + t^ 


t'+t^ 


t + t^ 




1 



Table 5. KnK{q,t)' for degree (3|2). 





(3,0; ) 


(2,1;) 


(2,0;1) 


(1,0; 2) 


(1,0; 1,1) 


(3,0; ) 


1 


q 


q + q'' 


q' 


q' 


(2,1;) 


qt 


1 


q + qH 


q-^t 


q' 


(2,0;1) 


t 


t 


l + qt 


q 


q 


(1,0;2) 




qt-^ 


t + qi^ 


1 


qt 


(1,0; 1,1) 


t^ 




t + i^ 


t 


1 



7.2. Two new conjectures for the Kostka coefficients. We conclude this section with the formula- 
tion of two remarkable conjectures that relate the generalized coefficients KQ\{q,t) of fermionic degree 
TO = 1 to the usual {q, t)-Kostka coefficients. 

Conjecture 25. Let A be a superpartition of fermionic degree m = l, and let J\ be the integral form of 
the Macdonald superpolynomial. Let also tj} be the linear application that maps Sn to Sq® . Then 

4'{Ja) = Ja@. (7.14) 

This conjecture implies that the usual (g, t)-Kostka coefficient K^\{q,t) of two partitions /i, A can be 
calculated from their lower-degree super-relatives as 



K^,^{q,t) 



E 

O I 0®=p 



KnA{q,t) 



(7.15) 



where A is any superpartition that can be obtained from A by replacing a square by a circle and the sum 
is over all fi's that can be obtained from fi by replacing a square by a circle. Moreover, the expression 
for the sum on the right-hand side is independent of the choice of A. We thus relate a Kostka coefficient 
of degree {n\0) to a sum of degree (n — 1|1) Kostka coefficients. In other words, the (n — 1|1) Kostka 
coefficients provide a refinement of the usual {q, t)-Kostka coefficients. 

For instance, consider A = /i = (3, 1): there are two ways of replacing a square by a circle: 



D 



Choosing A — (2; 1), we have [7]: 

J{2;1) = q'^tS(^0;3)+q'^{qt+^)S{0-2,l)+q^S^0;l, 



(J 

-q{qt + l)S 1^1.2) +qS{i;i,i) 



(7.16) 



-iqt + l)S(2;i)+tS^3;o) (7.17) 
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Table 6. Knh{q,t)' for degree (4|1). 





(4;) 


(0;4) 


(3;l) 


(i;3) 


(0;3,1) 


(2; 2) 


(4 


) 


1 




<j + + 


q^ +q'' + gt> 


g^ + g'' + q' 


q' + Q-^ 


(0 


4) 


t 


1 


qt + q^t + q-H 


g + g^ + q-H 


q + q'^ + q^ 


qH + qH 


(3 


1) 


t 




1 + gi + g^t 


q^ + q-^t + qH 


g^ + qH + qH 


q + qH 


(1 


3) 




qt 


t + qi'' + q'^t^ 


1 + q-^t + g^t^ 


q + qH + q-H 


qt + qH'' 


(0 


3,1) 




t 


i'' + qi-^ + q^t'^ 


t + gt + g^t^ 


1 + gt + qH 


qt^ + g^i^ 


(2 


2) 




q^t^ 


t + qt + qi-' 


gi + q^t + q-^t^ 


q^t + q-H + q-H'^ 


1 + q^i-' 


(0 


2,2) 






+ gt^ + qf^ 


t + qt^ + qt^ 


t + qt + qi^ 


t^ + q'H-^ 


(2 


1,1) 




qh^ 


t + i-' + qt^ 


qt + q'^t'-' + q-'t^ 


q^t + q^i^ + q-H'^ 


t + qt^ 


(1 


2,1) 




qt^ 


+ + gt"* 


t + qt^ + g^f* 


qt + qt'' + qH^ 


+ qt-'' 


(0 


2,1,1) 






+ + gt6 


t-^ +t'^ + qf^ 


t + t'^ + qt'^ 


t^ + gt^ 


(1 


1^) 






t-^ + t"* + i" 


i-^ + gt"* + gt'' 


qt^ + qf^ + qt'' 


i'+t^ 


(0 


1^) 








t* + t" + t ' 


t'' 


t'' +t>' 





(0;2,2) 


(2; 1,1) 


(i;2,i) 


(0;2,1,1) 


(i;i^) 


(0; 14) 


(4 


) 


g^ + g** 


g=^ + g"* + g^ 


g" + g5 + g^ 


g'^ + g** + g** 


g« 


glO 


(0 


4) 


q' + q^ 


q-H + q'^t + g^t 


g^ + q'^t + q^t 


g^ + g"* + q^ 


q^t 


g« 


(3 


1) 


q* + qH 


g + g^ + q-^t 


q^ + g-' + qH 


g* + g-' + q^t 


q' 




(1 


3) 


q^ + q-H 


qt + q^t + q-^i^ 


q + qH + qH^ 


q' + g-^ + qH 


q-'t 


q' 


(0 


3,1) 


q + qH 


qt^ + q^i^ + q-H-^ 


qt + g^t^ + q-H''' 


q + q'' + qH 


q-'i^ 


q' 


(2 


2) 


q^ + qH-' 


q + qt + q'H 


q + qH + q-H 


q'-^ + q'-H. + qH 


q' 


q' 


(0 


2,2) 


1 + q't' 


qi'' + gt^ + g^i^ 


qt + qi'' + q^i-'- 


q + qt + qH 


qH^ 


q' 


(2 


1,1) 


q^t + q-H'^ 


1 + gt + qt'^ 


q + qt + q'H''' 


q^ + q'-'t + q^i' 


q 


q' 


(1 


2,1) 


qt + qH^ 


t + qt'' + qt-^ 


1 + gt^ + q'H'-' 


q + q'H + q'H''' 


qt 


q' 


(0 


2,1,1) 


t + qt^ 


i-^ + gt" + qt^ 


i'' + qi' + qt* 


l + qt + qi^ 


qi-' 


q 


(1 


1^) 


qt^ + qf^ 


t + t-' + t-^ 


t + i'' + qi' 


qt + qi' + qi' 


1 


q 


(0 


1^) 


t^ + f^ 


+ t"^ + 


t^ +t" + 


t + i'' +t'-' 


t"^ 


1 



Table 7. KnA{q,ty for degree (4|2). 





(4,0;) 


(3,1;) 


(3,0;1) 


(1,0; 3) 


(2,0; 2) 


(2,l;l) 


(2,0; 1, 1) 


(1,0; 2,1) 


(1,0; l3) 


(4,0;) 


1 


g + g^ 


g + g^ + g^ 


q' 


q'-^q" 


q' 


q-^ +q*+ q" 


g^ + g'' 


q" 


(3,1;) 


qt 


1 + g^t 


g + g^t + q'-'t 


q't 


q' + q-'t 


q 


q'' +q-' + q*t 


g^ + q^t 


q' 


(3,0; 1) 


t 


t + qt 


1 + gt + q't 


q' 


q + qH 


qt 


q + q' + q'-'t 


q' +g^ 


q' 


(1,0; 3) 


i^ 


qt'^ + q'i' 


t + qt'^ + q'^i' 


1 


qt + q'^t'^ 


q'-'i-' 


qt + q'^t + q^t'^ 


q + q'H 


q-'t 


(2,0; 2) 


i^ 


t + qi'' 


t + qt + qi' 


qt 


1 + q'H'^ 


qt 


q + qt + q't 


q + qH 


q' 


(2,i;i) 


qi' 


t + qi' 


qt + qi' + q'i' 


q-'i-' 


qt + q''i'' 


1 


q + q't + q'i' 


q''t + q-'i' 


q' 


(2,0; 1, 1) 


t^ 


i^+i-^ 


t + i' + qi-' 


qt 


t + qi' 


i^ 


1 + gi + qi' 


q + qt 


q 


(1,0; 2,1) 


t^ 


i' + qt^ 


i'' +i-' + qt* 


t 


i' + qi' 


qti 


t + qi' + qi-' 


l + qi' 


qt 


(1,0; 1^) 


t" 


t^ + i- 


i' + t* + 1" 


i-i 


t'^+t^ 


ii 


t + i' +t}' 


t + i' 


1 



SO that, using (|7.15p : 

K{2-1) (2;1) + -^(0:3) (2;1) = (1 + 9*) + = -^(3,1) (3,1)- (7-18) 

The same result foUows by taking A = (0; 3): 

>^(0;3) = 'S'(0;3) +g(g + l)'5(0;2a) +9^'S'(0;ia,l) + (7(qt + l)S'(i;2) +(?t((7 + l)5(2;i) +t5(3;0) (7.19) 

imphes that 

-^(2:1) (0;3) + ■f^(0;3) (0;3) = + Q^t) + 1 = -^^^(3,1) (3,1)- (7-20) 

To formulate the second conjecture, we need to introduce the notion of a concatenable superpartition, 
defined as one for which > K\. Such a superpartition can be transformed into a partition A by 
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removing the semi-coma: A = (A°, A'*). For instance, (5, 3; 2, 1, 1) is concatenable and the corresponding 
partition is A = (5, 3, 2, 1, 1). 

Conjecture 26. Let A be a concatenable superpartition of fermionic degree m — \, and let A be its 
corresponding partition. Let also (j) be the linear application that maps Sq to 5*^ if fl is concatenable ( and 
II its correspoding partition) and to zero otherwise. Then 

0( Ja) = Jx (7.21) 



If A and are two m = 1 concatenable superpartitions whose corresponding partitions are A and /i, 
this imphes that: 

KAn^Kx^. (7.22) 
For instance, considering 4>{J(2-i)) given in (|7.17p . one recovers the expression of J(2,i)' 

J{2,i) = 9'5'(i,i4) + {qt + 1)S'(24) + ^^(3)- (7.23) 
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Appendix A. Proof of a combinatorial identity 



The equivalence, as N goes to oo, between the two norms in ()6.23p relies on the equality 

/iA= h\ 



lim 

AT— )-CXD 



"a 



(A.1) 



i(A»,A"-") 

The limit needs to appear on the right-hand-side since there is a residual dependence upon N in the ratio 
(iAs«/'^(A».As«) (the reversed partition A*''^ contains N — m — i?(A*) zeros). 

We first introduce, as in [TBI, a convenient decomposition of the leg-lengths of a composition 7: 

l{s) = l^s) + l\s) (A.2) 

with 

l^s) = #{fc = 1, z - 1 |j < 7fc + 1 < 7^} 

l\s) = #{fc = I + 1, N I i < 7fc < l^} (A.3) 

In order to better visualize expressions Z^(s) and l^{s), we put (as in [19]) a symbol at the end of each 
row of 7, here trading the French hexagon for a triangle, e.g., 

A 

A 



7= (0,0,1,3,3) 





A 










A 








A 



(A.4) 



For s = {i,j), l'^{s) is given by the number of triangles above row i in columns j' for j < j' < 7^ (e.g., 
Z^((4, 1)) = 3 in the above example), while l'^{s) is given by the number of triangles below row i in columns 
j' for j + 1 < j' < 7i + 1 (e.g., ^"^((4, 1)) = 1 in the above example). 

Lemma 27. Ldentity (jA.ip is equivalent to 



d', 



( A°«,A°) 
/lAa 



"A'-" 
^(A",A=«) 



hi 
hi' 



(A.5) 
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where 



^(A»,A»«) 



n 



J 



(A.6) 



that is, the products corresponding the the cells in the first columns of were removed from d/^sn/d^^/^a a^^)- 
Proof. Wc first isolate the part of dAs-n/d^Ao^A^^) that depends upon N: 



lim 

Af->oo a, 



d^sR 



(A'',A=-«) 



Xa<.« 



^(A^.A'-R) 



lim TT 
Ar->-oo J--'- 



(i,i)eA= 



'■(A»,A=«) 



R)(S)' 



It thus suffices to prove that 



lim I I 



(i4)eA' 



C?A---"(s) 



= 1 



(A.7) 



(A., 



The last equality is rather clear: it is a ratio of terms of the form 1 — for which the leg-length 

l{s) tends to infinity as N goes to infinity (the number of triangles above (i, 1) G A^ goes to infinity). 
Take for instance the case A = (2, 1; 2, 1) and introduce M zeros represented by A*^: 

A 

A^ 

(o*^l,2) 



(2,l,0*^l,2) : A 









A 





A 









A 







A 



(A.9) 



A 



The ratio on the left-hand-side becomes 

df,sR{s) 



lim 

M-s-oo 



n 



= lini Tz ^ . J , OS = 1- 



(i4)eA= 



i(A< 



A=R)(s) M^oo (1 - gtA^+l)(l - 92tM+3) 



(A.IO) 

□ 



Proposition 28. Identity (jA.5|) holds. 



Proof. First, observe that the identity (jA.l[) is satisfied identically (for all N actually) when A° = and 
A'* = II, since 



, , d' d,,R 

l.h.s. JAU = lim -^^7-^ = 



N- 



d^R 



and r.h.s. (|A31) = ^ = ^ = 



(A.ll) 



The result is thus true in the case A° = 0. We thus suppose by induction that (jA.5[) holds for some 
A — {A'^; A**) and the aim is to prove that (jA.Sp is still valid if we add a fermionic row > A" to obtain 
A = (A"; A"), where A" (&, A?, Ag, . . . )• Defining 

F(A) 



AF(A) 



F(A) 



it thus suffices to demonstrate that 



A l.h.s. (|X5|) = A r.h.s. dASJ • 



(A.12) 



(A.13) 



Given that A'' = A'', the factors dj^sR and Ha^ are not affected by the transformation A — ^ A. Hence 



Al.h.s. dXll = A 



'^(A°«,A°) 



Finaly, (IA.13|) will follow from the two relations 

(1) : Ad'(^„«^^.) =Ahl and (2) : A [d(Ao,A=«)]o 



Allbarh^^ 



(A.14) 



(A.15) 
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where h^^ is defined as h^^ except that the product runs over all squares of A: 

[|(l-q°A.(«)+4iAeW) and = J|(l-g''AeMi'A.M+i). (A.16) 

sSA s£A 

Observe that, 



hi_hi 



(A.17) 



"■A "-A 

and thus, as claimed, we only have to prove the two relations (jA.15|) . 

There are two types of contributions to A: those corresponding to the modifications to the leg- lengths 
of the squares of A'' in rows of length larger than 6, denoted by Ai, and those corresponding to the 
squares of the added fermionic row, denoted by A2. They will be treated separately. 

Consider first the variation Ai and the relation (1). In this case, d'^ — ?■ where 7 = (A"''^, A*) = and 

7 = (A"^, 6, A**) = ((A A*). The leg-lenghts of the squares in columns 1 < j < 6 + 1 and rows 7,; > 6 
is increased by 1 by adding the new fermionic row. We have thus: 

< , , ^ (A-18) 

1 - ga^(s) + l^i-,(s) \ ' 

li>b 
l<j<b+l 

where we indicated explicitly with respect to which diagram the arm- and leg-lengths are calculated. 
Consider now the corresponding variation of . Since the expression of involves Z^y®, the addition of 
the fermionic row of size b increases by 1 the leg-length of the squares in column 1 < j < & + 1 of rows 
Af > h: 

Aih\= n — - — T\ — r^^- (A.19) 

s=(i,j)eA= ^ 

A|>b 
l<j<b+l 

In order to compare expressions (|A.18I) and (|A.19I) , we need to clarify the meaning of the entries in each 
product. Note that in the first product, the rows % > b are such that ji — Af, for some i' . The product 
is thus over the same number of squares in the two cases. Now, in reordering the rows of 7 to get A* 
(an operation that does not affect the arm- lengths), we readily see that a-y = oa* in (|A.18|) . Next, to 
compare the leg-lengths, we note that for s = the leg-length l^{s) — li^{s) + l'l^{s) in (|A.18[) is such 

that 

l}i = #{k>z'\j <Al} (A.20) 

and 

= #{fc < « I j<lk + l< l^} = I J < A^ + 1} . (A.21) 

The sum -|- ij:^ thus corrresponds exactly to the definition of in (lA.lQp . This demonstrates the 
equivalence of relations (lA.lSp and (|A.19p . 

Consider next relation (2), focusing again on the contribution Ai. Let us first obtain the variation 
resulting from — dfj, where i] = (A°,A^^) and fj — (6,A°,A'*^). The analysis of Aid^ is similar to 
the one above for Aid'^, except that we need to keep in mind that the contribution of the first column 
is not considered anymore. We find 



rii>b 
2<j<b+l 



Let us now turn to the corrresponding variation in h^. The expression for involves a^® which is the 
same as ca* for a bosonic row. Also, since the leg-length entering in is l\* , which does not count the 
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possible circle at the end of the column, the addition of the fcrmionic row of length b increases by 1 the 
leg- lengths of the squares in columns I < j < b (and not b + 1) of the rows A| > b: 

1 „aA'(s)+'A*(s)+2 

^1^1= TT , w r,^, - (A.23) 

s=(jj)eA= 

A=>6 
l<i<6 

Before we can do a direct comparison between (IA.22I) and (jA.23p . we have to perform the substitution 
j — > j + 1 in (jA.22|) . Since a^{i,j + 1) + 1 = a,f{i,j), this gives 

s={i,j+i)ev 

Vi>b 
l<j<b 

Letting iji = A*,, we see that the powers of q are identical in both expressions. Moreover, since the block 
A" still lies in the upper position of the composition, we can write the leg- length (i, j -I- 1) using eq. 
(|X3|) as: 

^^,J + 1) = I.? <An + #{k > z' I J < Al < Af,} + #{fc < i' I Al - Af,} (A.25) 

which corresponds, up to a reordering of the parts of A* of the same size as A|, , precisely to the expression 
of /a* in (IA.23p . We have thus verified the equivalence of (|A.22|) and (|A.23|) . 

Consider now the variation A2, namely, the contribution of the squares of the added fermionic row of 
size b. As before, we let 7 = (A"^^, A'*), 7 = (A'^^, 6, A") and A = (A"^; A^) with A"^ = (&, AJ, Af , . . . )■ If 
the added fermionic row of size b corresponds to 7^ (resp. A^/) in 7 (resp. A), the verification of relation 

(1) amounts to compare 

A2d;= Jl [l-q''-^+^M^'^] and H [1 - ^^''^'^l • (A.26) 

l<j<b l<j<b 

The powers of q are manifestly the same in the two contributions. The expression for the leg-length on 
the l.h.s. is 

h{i,j)^lHi,j)+l\i,j), where l^z,j) = i^{k\j<Al + l} and lHhj)^#{k\j<Ai<b} 

(A.27) 

This form of mi,j) is clearly the same as the expression of on the r.h.s, which demonstrates 

the equivalence of the two variations in (jA.26P . 

It only remains to establish the correctness of relation (2) under A2. Recall that rj = (A°, A^^) and 
fj = (fe. A", A'*^). If the added fermionic row of size b corresponds to A^ in A, the verification of relation 

(2) amounts to compare 

^2^,,= \{ [I - q''-^+^t^'^'^^^^'^+^] and A2 4= \{[l-q'''^+^t^'^'''''^'^'^^] (A.28) 

l<i<b l<j<b 

where we used the fact that a^g, = a^^, + 1 given that row i of A is fermionic. Again the powers 
of q match. We have 

= = #{k I ]<Kl<boT3< Al < b} (A.29) 

which corresponds to on the r.h.s. This completes the demonstration of the two relations (jA.lSp 

and thus of Proposition [28l □ 



Let us consider an example: A — (0; 4, 1) and A ~ (2, 0; 4, 1). We have thus added a fermionic row of 
length 2. We ignore the zeros of A* given that they contribute only to the first column which is removed. 
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The triangle at the end of the added row is written in parenthesis and the circle of the added row is 
marked by a x : 

A 



7 = (0,2, 4,1) 







(A) 












A 



A 



77= (2,0,1,4) 







(A) 


A 


A 













A =(2,0; 4,1) 



A 



We have thus, for the product of the two variations Ai and A2: 

^(0,2,4,1) _ jl - qH^){l ~ q't)il - qH) ^ , , _ ^(2,0;44) 



d[oA,l) {l-qH^)a-q'){l-q') 
rf(2,04,4) (1 - qH^){l - qH^) 



(0:4,1) 



rf(0,i,4) {I ~ qH^)il - qH) 



X {l-qH)il^q) 



"■(2,0:4,1) 
(0;4,1) 



(A.30) 



(A.31) 
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